-- Logs begin at Wed 2022-08-10 07:45:15 UTC, end at Wed 2022-08-10 09:34:49 UTC. -- Aug 10 08:22:05.870048 dv-cinder-780759-5-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 10 08:22:05.878800 dv-cinder-780759-5-iscsi devstack@keystone.service[80449]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 10 08:22:05.880997 dv-cinder-780759-5-iscsi devstack@keystone.service[80449]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Aug 10 08:22:05 2022] *** Aug 10 08:22:05.880997 dv-cinder-780759-5-iscsi devstack@keystone.service[80449]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 10 08:22:05.880997 dv-cinder-780759-5-iscsi devstack@keystone.service[80449]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 10 08:22:05.880997 dv-cinder-780759-5-iscsi devstack@keystone.service[80449]: nodename: dv-cinder-780759-5-iscsi Aug 10 08:22:05.880997 dv-cinder-780759-5-iscsi devstack@keystone.service[80449]: machine: x86_64 Aug 10 08:22:05.880997 dv-cinder-780759-5-iscsi devstack@keystone.service[80449]: clock source: unix Aug 10 08:22:05.880997 dv-cinder-780759-5-iscsi devstack@keystone.service[80449]: pcre jit disabled Aug 10 08:22:05.880997 dv-cinder-780759-5-iscsi devstack@keystone.service[80449]: detected number of CPU cores: 4 Aug 10 08:22:05.880997 dv-cinder-780759-5-iscsi devstack@keystone.service[80449]: current working directory: / Aug 10 08:22:05.880997 dv-cinder-780759-5-iscsi devstack@keystone.service[80449]: detected binary path: /usr/bin/uwsgi-core Aug 10 08:22:05.880997 dv-cinder-780759-5-iscsi devstack@keystone.service[80449]: your processes number limit is 47680 Aug 10 08:22:05.880997 dv-cinder-780759-5-iscsi devstack@keystone.service[80449]: your memory page size is 4096 bytes Aug 10 08:22:05.880997 dv-cinder-780759-5-iscsi devstack@keystone.service[80449]: detected max file descriptor number: 2048 Aug 10 08:22:05.880997 dv-cinder-780759-5-iscsi devstack@keystone.service[80449]: lock engine: pthread robust mutexes Aug 10 08:22:05.882013 dv-cinder-780759-5-iscsi devstack@keystone.service[80449]: thunder lock: enabled Aug 10 08:22:05.882013 dv-cinder-780759-5-iscsi devstack@keystone.service[80449]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 10 08:22:05.882013 dv-cinder-780759-5-iscsi devstack@keystone.service[80449]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 10 08:22:05.924413 dv-cinder-780759-5-iscsi devstack@keystone.service[80449]: Python main interpreter initialized at 0x5593678b59c0 Aug 10 08:22:05.924413 dv-cinder-780759-5-iscsi devstack@keystone.service[80449]: python threads support enabled Aug 10 08:22:05.924413 dv-cinder-780759-5-iscsi devstack@keystone.service[80449]: your server socket listen backlog is limited to 100 connections Aug 10 08:22:05.924413 dv-cinder-780759-5-iscsi devstack@keystone.service[80449]: your mercy for graceful operations on workers is 90 seconds Aug 10 08:22:05.924829 dv-cinder-780759-5-iscsi devstack@keystone.service[80449]: mapped 403077 bytes (393 KB) for 2 cores Aug 10 08:22:05.924829 dv-cinder-780759-5-iscsi devstack@keystone.service[80449]: *** Operational MODE: preforking *** Aug 10 08:22:05.925142 dv-cinder-780759-5-iscsi devstack@keystone.service[80449]: *** uWSGI is running in multiple interpreter mode *** Aug 10 08:22:05.925142 dv-cinder-780759-5-iscsi devstack@keystone.service[80449]: spawned uWSGI master process (pid: 80449) Aug 10 08:22:05.925366 dv-cinder-780759-5-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 10 08:22:05.926228 dv-cinder-780759-5-iscsi devstack@keystone.service[80449]: spawned uWSGI worker 1 (pid: 80461, cores: 1) Aug 10 08:22:05.926228 dv-cinder-780759-5-iscsi devstack@keystone.service[80449]: spawned uWSGI worker 2 (pid: 80462, cores: 1) Aug 10 08:22:05.926395 dv-cinder-780759-5-iscsi devstack@keystone.service[80449]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 10 08:22:11.446781 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 10 08:22:11.446781 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: from cryptography.utils import int_from_bytes Aug 10 08:22:11.447489 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 10 08:22:11.447489 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: from cryptography.utils import int_from_bytes Aug 10 08:22:11.447900 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 10 08:22:11.447900 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: from cryptography.utils import int_from_bytes Aug 10 08:22:11.448020 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 10 08:22:11.448020 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: from cryptography.utils import int_from_bytes Aug 10 08:22:11.498378 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 10 08:22:11.498629 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 10 08:22:11.498885 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] command line args: [] {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 10 08:22:11.499248 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 10 08:22:11.499496 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ================================================================================ {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 10 08:22:11.499759 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 10 08:22:11.499858 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] admin_token = **** {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 08:22:11.499933 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 10 08:22:11.500123 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] command line args: [] {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 10 08:22:11.500216 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] config_dir = [] {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 08:22:11.500379 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 08:22:11.500466 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 10 08:22:11.500548 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] ================================================================================ {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 10 08:22:11.500627 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] config_source = [] {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 08:22:11.500845 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] admin_token = **** {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 08:22:11.500984 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] debug = True {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 08:22:11.501149 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 08:22:11.501414 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] config_dir = [] {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 08:22:11.501414 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 08:22:11.501604 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] config_source = [] {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 08:22:11.501696 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 08:22:11.501796 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] debug = True {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 08:22:11.501940 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 08:22:11.502066 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 08:22:11.502441 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 08:22:11.502566 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 08:22:11.502667 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 08:22:11.502780 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 08:22:11.502886 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 08:22:11.503069 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] list_limit = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 08:22:11.503182 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 08:22:11.503289 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] log_config_append = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 08:22:11.503354 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] list_limit = None {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 08:22:11.503413 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 08:22:11.503548 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] log_config_append = None {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 08:22:11.503635 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] log_dir = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 08:22:11.503764 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 08:22:11.503853 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] log_file = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 08:22:11.503957 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] log_dir = None {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 08:22:11.504161 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] log_file = None {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 08:22:11.504247 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 08:22:11.504453 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 08:22:11.504533 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 08:22:11.504756 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 08:22:11.504930 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 08:22:11.505087 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 08:22:11.505348 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 08:22:11.505441 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 08:22:11.505548 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 08:22:11.505636 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 08:22:11.505718 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 08:22:11.505823 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 08:22:11.505987 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 08:22:11.506201 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 08:22:11.506299 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 08:22:11.506533 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 08:22:11.506612 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 08:22:11.506823 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 08:22:11.506917 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 08:22:11.507114 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 08:22:11.507206 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 08:22:11.507407 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 08:22:11.507530 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 08:22:11.507681 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 08:22:11.507773 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 08:22:11.507967 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 08:22:11.508057 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 08:22:11.508773 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 08:22:11.509042 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 08:22:11.509297 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 08:22:11.509620 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.57/identity {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 08:22:11.509820 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] publish_errors = False {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 08:22:11.509901 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.57/identity {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 08:22:11.510024 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] publish_errors = False {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 08:22:11.510167 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 08:22:11.510246 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 08:22:11.510384 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 08:22:11.510465 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 08:22:11.510590 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 08:22:11.510679 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 08:22:11.510879 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 08:22:11.510977 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 08:22:11.511117 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 08:22:11.511198 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 08:22:11.511354 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] standard_threads = False {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 08:22:11.511436 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] standard_threads = False {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 08:22:11.511588 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 08:22:11.511724 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 08:22:11.511832 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 08:22:11.512008 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 08:22:11.512096 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 08:22:11.512177 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] use_journal = False {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 08:22:11.512252 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 08:22:11.512346 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] use_json = False {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 08:22:11.512426 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] use_journal = False {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 08:22:11.512507 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] use_stderr = False {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 08:22:11.512589 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] use_json = False {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 08:22:11.512664 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] use_syslog = False {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 08:22:11.512792 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] use_stderr = False {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 08:22:11.512900 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 08:22:11.513007 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] use_syslog = False {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 08:22:11.513179 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.513337 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 08:22:11.513414 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.513596 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.513685 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.513891 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.513973 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.514137 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.514244 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.514377 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.514556 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.514643 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.514849 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.514937 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.515096 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.515178 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.515344 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.515539 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.515619 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.515818 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.515965 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.516068 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.516305 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.516391 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.516555 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.516647 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.516800 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.516952 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.517058 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.517150 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.517308 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.517391 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.517546 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.517627 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.517793 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.517872 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.518033 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.518122 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.518267 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.518347 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.518523 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.518603 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.518774 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.518855 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.519018 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.519105 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.519252 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.519332 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.519478 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.519553 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.519705 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.519786 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.519942 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.520023 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.520211 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.520334 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.520423 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.520498 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.520656 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.520742 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.520886 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.521029 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.521146 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.521234 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.521392 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.521472 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.521631 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.521713 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.521860 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.521945 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.522091 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.522178 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.522326 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.522412 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.522562 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.522649 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.522794 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.522901 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.523009 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.523108 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.523243 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.523389 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.523507 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.523622 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.523732 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.523834 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.524569 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.524662 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.524811 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.524985 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.525082 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.525159 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.525292 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.525374 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.525532 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.525751 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.525751 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.525921 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.525998 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.526119 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.526222 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.526353 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.526434 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.526562 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.526672 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.526755 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.526908 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.527046 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.527147 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.527238 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.527376 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.527505 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.527586 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.527660 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.527838 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.527932 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.528073 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.528155 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.528307 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.528390 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.528541 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.528623 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.528753 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.528831 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.529037 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.529136 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.529271 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.529360 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.529507 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.529587 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.529739 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.529830 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.529996 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.530087 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.530232 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.530314 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.530464 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.530545 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.530700 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.530790 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.530934 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.531013 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.531171 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.531260 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.531408 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.531564 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.531643 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.531715 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.531880 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.531960 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.532113 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.532202 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.532352 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.532432 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.532585 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.532757 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.532852 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.532977 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.533070 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.533148 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.533291 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.533373 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.533525 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.533613 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.533760 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.533842 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.533998 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.534079 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.534232 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.534314 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.534466 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.534557 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.534700 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.534789 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.534936 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.535017 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.535173 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.535256 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.535409 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.535497 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.535643 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.535725 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.535874 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.535957 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.536106 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.536187 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.536338 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.536418 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.536566 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.536647 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.536793 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.536874 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.537047 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.537131 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.537277 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.537361 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.537510 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.537601 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.537739 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.537821 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.537979 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.538062 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.538212 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.538293 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.538456 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.538545 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] auth.external = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.538687 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.538768 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.538937 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.539025 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.539193 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.539274 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.539443 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] auth.password = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.539576 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.539683 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] auth.external = None {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.539768 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] auth.token = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.539894 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.539983 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.540142 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.540222 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.540390 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.540480 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.540624 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.540706 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] auth.password = None {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.540851 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] auth.token = None {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.540978 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.541116 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.541208 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.541351 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.541434 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.541587 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] credential.caching = True {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.541669 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.541816 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.541904 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.542067 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.542150 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.542317 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.542398 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.542548 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.542631 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.542784 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] credential.caching = True {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.542866 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.543016 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.543103 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.543265 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.543347 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.543511 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.543593 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.543747 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.543835 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.543988 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.544069 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.544224 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.544313 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.544489 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.544579 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.544741 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.544823 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.545000 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.545089 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.545255 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.545413 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.545494 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] federation.caching = True {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.545568 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.545712 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.545796 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.545959 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.546049 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.546205 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.546288 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.546437 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.546520 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.546684 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.546775 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] federation.caching = True {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.546922 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.547005 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.547166 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.547247 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.547402 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.547491 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.547612 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.547696 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.547862 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.547945 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.548096 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.548191 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.548333 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] identity.caching = True {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.548414 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.548569 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.548650 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.548812 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.548902 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.549071 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.549156 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.549310 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.549392 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.549543 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] identity.caching = True {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.549636 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.549778 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.549859 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.550023 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.550105 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.550256 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.550346 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.550493 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.550577 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.550733 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.550814 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.550963 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.551054 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.551193 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.551274 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.551438 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.551527 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.551683 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.551773 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.551911 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.551997 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.552142 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.552225 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.552378 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.552469 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.552622 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.552705 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.552865 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.552968 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.553136 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.553232 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.553367 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.553448 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.553605 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.553688 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.553836 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.553920 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.554071 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.554153 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.554312 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.554401 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.554548 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.554630 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.554784 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.554867 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.555015 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.555095 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.555286 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.555368 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.555533 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.555755 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.555838 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.556002 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.556091 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.556249 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.556331 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.556481 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.556563 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.556720 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.556814 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.556977 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.557065 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.557217 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.557299 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.557454 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.557545 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.557696 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.557778 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.557925 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.558006 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.558161 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.558250 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.558390 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.558471 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.558628 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.558710 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.558865 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.558955 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.559098 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.559180 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.559333 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.559414 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.559570 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.559664 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.559807 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.559888 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.560050 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.560132 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.560286 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.560375 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.560516 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.user = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.560597 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.560756 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.560839 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.561026 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.561122 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.561266 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.561348 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.561502 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.561584 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.561738 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.561828 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.561967 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] ldap.user = None {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.562050 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.562206 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.562286 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.562439 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.562534 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.562676 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.562759 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.562911 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.563000 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.563149 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.563240 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.563379 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.563462 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.563610 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.563691 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.563839 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.563930 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.564073 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.564155 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.564306 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.564388 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.564541 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.564632 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.564770 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.564852 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.565036 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.565120 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.565275 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.565370 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.565512 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.565595 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.565748 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.565831 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.565984 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.566074 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.566211 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.566292 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.566456 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.566538 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.566691 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.566780 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.566925 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.567005 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.567159 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.567242 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.567393 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.567483 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.567626 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.567707 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.567862 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.567944 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.568095 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.568173 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.568335 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.568417 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.568563 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.568644 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.568799 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.568881 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.569055 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.569137 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.569291 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.569373 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] resource.caching = True {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.569527 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.569611 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.569771 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.569851 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.570008 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.570088 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.570238 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.570321 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.570476 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.570558 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.571233 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.571321 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] resource.caching = True {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.571484 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.571665 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.571748 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.571951 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.572078 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.572187 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] role.caching = True {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.572324 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.572434 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] role.driver = sql {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.572577 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.572659 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.572836 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.572936 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.573098 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.573183 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.573344 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.573426 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.573582 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.573672 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.573822 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.573905 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.574055 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] role.caching = True {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.574138 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.574300 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] role.driver = sql {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.574383 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.574531 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.574614 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.574778 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.574862 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.575021 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.575102 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.575265 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.575348 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.575508 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.575589 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.575751 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.575832 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.575992 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.576144 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.576227 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.576381 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.576464 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.576631 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.576722 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.576881 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.577061 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.577179 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.577261 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.577426 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.577510 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.577668 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.577751 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.577964 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.578066 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.578217 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.578301 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.578458 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.578540 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.578692 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.578775 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.578926 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.579114 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.579197 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.579365 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.579448 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.579602 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.579693 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.579856 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.579940 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.580097 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.580178 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.580335 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.580425 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.580571 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.580679 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.580800 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.580879 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] token.caching = True {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.581059 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.581155 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.581309 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.581390 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.581545 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.581626 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.581795 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.581889 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.582032 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.582117 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.582268 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.582349 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.582496 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.582585 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.582734 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] token.caching = True {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.582817 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.582971 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.583054 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.583207 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.583296 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.583439 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.583520 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.583674 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.583755 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.583926 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.584016 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.584170 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.584252 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.584400 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.584481 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.584644 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.584734 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.584889 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.585005 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.585151 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.585234 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] database.connection = **** {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.585389 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.585482 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.585624 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.585707 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.585870 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.585952 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.586116 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.586199 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.586891 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.587009 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.587148 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.587305 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.587389 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.587600 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.587691 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.587845 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.587927 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] database.connection = **** {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.588082 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.588175 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.588327 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.588410 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.588557 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.588635 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.588792 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.588876 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.589051 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.589135 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.589291 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.589376 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.589520 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.589600 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.589754 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.589835 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.589986 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.590068 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.590145 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 10 08:22:11.590221 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.590451 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.590685 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.590929 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.591164 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.591409 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.591637 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.591875 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.592110 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 08:22:11.592278 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 10 08:22:11.619083 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80462) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 08:22:11.619310 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80462) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 08:22:11.619533 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80462) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 08:22:11.619842 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80461) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 08:22:11.620072 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80461) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 08:22:11.620296 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80461) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 08:22:11.629515 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80462) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 08:22:11.630079 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80461) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 08:22:11.658902 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80462) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 08:22:11.659032 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80461) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 08:22:11.659128 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80462) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 08:22:11.659209 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80461) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 08:22:11.710631 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80461) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 08:22:11.711158 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80462) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 08:22:11.761975 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80461) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 08:22:11.762200 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80461) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 08:22:11.763759 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80462) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 08:22:11.763996 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80462) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 08:22:11.829977 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80461) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 08:22:11.830200 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80461) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 08:22:11.830312 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80462) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 08:22:11.830423 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80461) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 08:22:11.830512 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80462) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 08:22:11.830641 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80461) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 08:22:11.830727 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80462) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 08:22:11.830856 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80461) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 08:22:11.830944 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80462) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 08:22:11.831069 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80461) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 08:22:11.831155 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80462) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 08:22:11.831328 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80462) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 08:22:11.853908 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80461) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 08:22:11.854134 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80461) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 08:22:11.854231 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80462) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 08:22:11.854355 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80461) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 08:22:11.854442 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80462) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 08:22:11.854569 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80461) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 08:22:11.854665 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80462) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 08:22:11.854780 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80461) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 08:22:11.854868 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80462) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 08:22:11.854990 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80461) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 08:22:11.855083 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80462) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 08:22:11.855200 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80461) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 08:22:11.855288 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80462) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 08:22:11.855419 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80462) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 08:22:11.866056 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80461) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 08:22:11.866392 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80462) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 08:22:11.879472 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80461) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 08:22:11.879699 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80461) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 08:22:11.879899 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80462) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 08:22:11.879988 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80461) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 08:22:11.880119 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80461) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 08:22:11.880204 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80462) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 08:22:11.880330 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80461) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 08:22:11.880418 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80462) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 08:22:11.880538 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80461) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 08:22:11.880625 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80462) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 08:22:11.880748 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80461) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 08:22:11.880845 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80462) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 08:22:11.880980 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80462) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 08:22:11.881194 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80462) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 08:22:11.898095 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:11.898302 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 10 08:22:11.898645 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:11.898911 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 10 08:22:11.899074 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:11.899284 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 10 08:22:11.899372 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:11.899596 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:11.899738 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:11.899867 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 10 08:22:11.899996 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:11.900124 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:11.900252 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:11.900392 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:11.900570 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:11.900829 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:11.911310 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:11.911543 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:11.911815 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:11.912105 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:11.912418 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:11.912739 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:11.913034 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:11.913318 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:11.919072 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=80461) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 08:22:11.920219 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=80462) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 08:22:11.922156 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=80461) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 08:22:11.922441 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:11.922730 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:11.923013 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:11.923301 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:11.923433 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=80462) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 08:22:11.923563 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:11.923692 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:11.923821 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:11.923969 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:11.924100 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:11.924229 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:11.924358 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:11.924486 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:11.924622 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:11.924752 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:11.924880 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:11.925032 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:11.925247 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:11.925519 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:11.925792 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:11.926070 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:11.948799 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:11.950311 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80461) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 08:22:11.950445 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:11.950592 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:11.951871 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80462) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 08:22:11.952159 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:11.955126 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=80461) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 08:22:11.955410 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:11.955691 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:11.956750 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=80462) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 08:22:11.957062 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:11.957348 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:11.962265 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=80461) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 08:22:11.962545 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:11.963973 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=80462) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 08:22:11.964262 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:11.966547 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80461) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 08:22:11.966838 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:11.967121 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:11.967405 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:11.967683 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:11.967961 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:11.968230 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:11.968360 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80462) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 08:22:11.968498 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:11.968629 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:11.968821 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:11.969134 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:11.969415 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:11.969694 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:11.969971 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:11.970243 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:11.983953 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:11.984230 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:11.986041 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:11.986325 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:11.988159 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80461) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 08:22:11.990279 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80462) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 08:22:11.991265 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:11.991560 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:11.994077 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:11.994375 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:11.996888 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80461) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 08:22:11.999032 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80462) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 08:22:12.000008 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80461) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 08:22:12.002236 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80462) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 08:22:12.003149 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:12.003449 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:12.003733 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:12.004024 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:12.004301 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:12.004589 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:12.005431 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:12.005735 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:12.006024 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:12.006322 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:12.006603 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:12.006890 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:12.021247 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80461) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 08:22:12.021534 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:12.021800 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:12.022063 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:12.023466 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80462) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 08:22:12.023760 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:12.024056 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:12.024323 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:12.029155 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:12.030707 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:12.030985 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:12.031273 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:12.032829 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:12.033132 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:12.033770 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=80461) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 08:22:12.034052 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:12.034350 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:12.034646 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:12.034935 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:12.035936 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=80462) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 08:22:12.036223 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:12.036525 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:12.036827 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:12.037134 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:12.046286 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=80461) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 08:22:12.046567 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:12.046847 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:12.047126 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:12.047405 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:12.047680 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:12.047954 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:12.048521 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=80462) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 08:22:12.048807 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:12.049120 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:12.049406 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:12.049690 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:12.049967 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:12.050242 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:12.065155 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=80461) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 08:22:12.067285 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=80462) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 08:22:12.068063 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80461) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 08:22:12.070255 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80462) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 08:22:12.071726 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:12.073909 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:12.074056 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:12.075473 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:12.075561 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=80461) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 08:22:12.075841 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:12.076129 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:12.076987 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=80462) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 08:22:12.077285 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:12.077569 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:12.082481 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:12.083959 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:12.084082 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=80461) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 08:22:12.085606 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=80462) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 08:22:12.087200 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:12.087488 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:12.087890 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:12.088327 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:12.088621 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:12.088927 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:12.089225 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:12.089504 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:12.098949 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80462) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 08:22:12.099241 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:12.099525 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:12.100354 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80461) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 08:22:12.100648 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:12.100949 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:12.107420 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=80462) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 08:22:12.107796 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:12.108180 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:12.108559 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:12.109048 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:12.109152 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=80461) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 08:22:12.109263 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:12.109349 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:12.109545 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:12.109667 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:12.109824 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:12.110011 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:12.110099 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:12.110361 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:12.110444 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:12.110632 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:12.110777 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:12.110897 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:12.111143 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:12.111276 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:12.111495 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:12.111590 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:12.111822 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:12.111906 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:12.112086 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:12.112259 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:12.112351 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:12.112610 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 08:22:12.146797 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x5593678b59c0 pid: 80461 (default app) Aug 10 08:22:12.148180 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x5593678b59c0 pid: 80462 (default app) Aug 10 08:22:12.150225 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ab4998-68fb-49c4-a6e4-ea4967090624 None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:12.150510 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ab4998-68fb-49c4-a6e4-ea4967090624 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:12.150779 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ab4998-68fb-49c4-a6e4-ea4967090624 None None] PATH_INFO: `/v3` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:12.155046 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1/1] 10.222.0.57 () {50 vars in 791 bytes} [Wed Aug 10 08:22:12 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:22:20.697688 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c9f326-68ea-4f76-a50e-8d8d66c72ef4 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:20.698261 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c9f326-68ea-4f76-a50e-8d8d66c72ef4 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:20.698261 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c9f326-68ea-4f76-a50e-8d8d66c72ef4 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:20.702646 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1/2] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:22:20 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:22:20.707730 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1cafb8-6856-48c9-a948-e5638d47e6ad None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:20.708121 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1cafb8-6856-48c9-a948-e5638d47e6ad None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:20.708294 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1cafb8-6856-48c9-a948-e5638d47e6ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:20.778847 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: INFO dbcounter [None req-ac1cafb8-6856-48c9-a948-e5638d47e6ad None None] Registered counter for database keystone Aug 10 08:22:20.798299 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG oslo_db.sqlalchemy.engines [None req-ac1cafb8-6856-48c9-a948-e5638d47e6ad None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80461) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 10 08:22:20.804326 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writer thread running {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 10 08:22:20.980958 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG passlib.handlers.bcrypt [None req-ac1cafb8-6856-48c9-a948-e5638d47e6ad None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80461) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 10 08:22:20.981313 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG passlib.handlers.bcrypt [None req-ac1cafb8-6856-48c9-a948-e5638d47e6ad None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80461) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 10 08:22:21.035125 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-ac1cafb8-6856-48c9-a948-e5638d47e6ad None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:22:21.036712 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-ac1cafb8-6856-48c9-a948-e5638d47e6ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:22:21.166797 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 2/3] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:22:20 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 460 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 10 08:22:21.313937 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c9f326-68ea-4f76-a50e-8d8d66c72ef4 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:21.383286 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: INFO dbcounter [None req-42c9f326-68ea-4f76-a50e-8d8d66c72ef4 None None] Registered counter for database keystone Aug 10 08:22:21.404608 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG oslo_db.sqlalchemy.engines [None req-42c9f326-68ea-4f76-a50e-8d8d66c72ef4 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80462) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 10 08:22:21.410917 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writer thread running {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 10 08:22:21.528420 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c9f326-68ea-4f76-a50e-8d8d66c72ef4 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:21.532569 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f7c82f-b789-40ec-b02c-65ac1421fbad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:21.533217 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f7c82f-b789-40ec-b02c-65ac1421fbad admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:21.533499 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f7c82f-b789-40ec-b02c-65ac1421fbad admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:21.533773 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f7c82f-b789-40ec-b02c-65ac1421fbad admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:21.542556 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f7c82f-b789-40ec-b02c-65ac1421fbad admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:21.594471 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: WARNING keystone.common.rbac_enforcer.enforcer [None req-15f7c82f-b789-40ec-b02c-65ac1421fbad admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 10 08:22:21.594757 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f7c82f-b789-40ec-b02c-65ac1421fbad admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:21.604812 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 2/4] 10.222.0.57 () {58 vars in 1324 bytes} [Wed Aug 10 08:22:21 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 08:22:21.610771 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1cafb8-6856-48c9-a948-e5638d47e6ad None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:21.625481 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1cafb8-6856-48c9-a948-e5638d47e6ad None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:21.629916 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7401e7-b95f-4b60-b791-fab5abb52467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:21.631012 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7401e7-b95f-4b60-b791-fab5abb52467 admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:21.631303 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7401e7-b95f-4b60-b791-fab5abb52467 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:21.631625 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7401e7-b95f-4b60-b791-fab5abb52467 admin admin] PATH_INFO: `/v3/projects/fe95c65d461a414fa25f77dc13eab47b` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:21.633091 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7401e7-b95f-4b60-b791-fab5abb52467 admin admin] RBAC: Authorizing `identity:get_project(project_id=fe95c65d461a414fa25f77dc13eab47b)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:21.684290 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: WARNING keystone.common.rbac_enforcer.enforcer [None req-0d7401e7-b95f-4b60-b791-fab5abb52467 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 10 08:22:21.684634 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7401e7-b95f-4b60-b791-fab5abb52467 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:21.689198 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 3/5] 10.222.0.57 () {56 vars in 1207 bytes} [Wed Aug 10 08:22:21 2022] GET /identity/v3/projects/fe95c65d461a414fa25f77dc13eab47b => generated 344 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:22:23.676343 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ba4b15-3f74-4abb-8f4a-1b84810d215d None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:23.676873 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ba4b15-3f74-4abb-8f4a-1b84810d215d None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:23.676873 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ba4b15-3f74-4abb-8f4a-1b84810d215d None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:23.680985 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 3/6] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:22:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 08:22:23.686958 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f6596d-0498-4aef-b6fc-8768cff50799 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:23.687407 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f6596d-0498-4aef-b6fc-8768cff50799 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:23.687495 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f6596d-0498-4aef-b6fc-8768cff50799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:23.708312 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-e9f6596d-0498-4aef-b6fc-8768cff50799 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:22:23.709828 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-e9f6596d-0498-4aef-b6fc-8768cff50799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:22:23.717616 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 4/7] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:22:23 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 10 08:22:23.739543 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ba4b15-3f74-4abb-8f4a-1b84810d215d None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:23.752519 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ba4b15-3f74-4abb-8f4a-1b84810d215d None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:23.756642 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b43bed-646e-48fb-9dc3-7b30c46a805f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:23.757260 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b43bed-646e-48fb-9dc3-7b30c46a805f admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:23.757541 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b43bed-646e-48fb-9dc3-7b30c46a805f admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:23.757815 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b43bed-646e-48fb-9dc3-7b30c46a805f admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:23.765396 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b43bed-646e-48fb-9dc3-7b30c46a805f admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:23.767165 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b43bed-646e-48fb-9dc3-7b30c46a805f admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:23.775880 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 4/8] 10.222.0.57 () {58 vars in 1324 bytes} [Wed Aug 10 08:22:23 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 08:22:23.781303 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f6596d-0498-4aef-b6fc-8768cff50799 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:23.791260 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f6596d-0498-4aef-b6fc-8768cff50799 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:23.795564 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6127a9d8-0c1a-47a7-b130-9177319c0f23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:23.796172 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6127a9d8-0c1a-47a7-b130-9177319c0f23 admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:23.796451 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6127a9d8-0c1a-47a7-b130-9177319c0f23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:23.796725 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6127a9d8-0c1a-47a7-b130-9177319c0f23 admin admin] PATH_INFO: `/v3/users/a79bffd3ad024a2f938d1f77cfd728d5` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:23.798072 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6127a9d8-0c1a-47a7-b130-9177319c0f23 admin admin] RBAC: Authorizing `identity:get_user(user_id=a79bffd3ad024a2f938d1f77cfd728d5)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:23.799880 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6127a9d8-0c1a-47a7-b130-9177319c0f23 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:23.801413 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 5/9] 10.222.0.57 () {56 vars in 1198 bytes} [Wed Aug 10 08:22:23 2022] GET /identity/v3/users/a79bffd3ad024a2f938d1f77cfd728d5 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:22:25.861841 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bbe950-8798-454e-a465-937aa908aaef None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:25.862413 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bbe950-8798-454e-a465-937aa908aaef None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:25.862413 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bbe950-8798-454e-a465-937aa908aaef None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:25.863715 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 5/10] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:22:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:22:25.869651 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe1270e-eaaa-4a6b-b2a4-7f00f2a40ec0 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:25.870086 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe1270e-eaaa-4a6b-b2a4-7f00f2a40ec0 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:25.870183 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe1270e-eaaa-4a6b-b2a4-7f00f2a40ec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:25.888062 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-5fe1270e-eaaa-4a6b-b2a4-7f00f2a40ec0 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:22:25.889243 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-5fe1270e-eaaa-4a6b-b2a4-7f00f2a40ec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:22:25.905775 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2902bb19-39d2-45ed-9270-07b7f26e5ed4 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:25.906060 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2902bb19-39d2-45ed-9270-07b7f26e5ed4 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:25.906334 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2902bb19-39d2-45ed-9270-07b7f26e5ed4 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:25.907655 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 6/11] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:22:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 08:22:25.912852 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a29415c-d13f-4741-9878-51f6d2cacefb None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:25.913157 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a29415c-d13f-4741-9878-51f6d2cacefb None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:25.913426 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a29415c-d13f-4741-9878-51f6d2cacefb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:25.957756 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG passlib.handlers.bcrypt [None req-5a29415c-d13f-4741-9878-51f6d2cacefb None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80462) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 10 08:22:25.958175 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG passlib.handlers.bcrypt [None req-5a29415c-d13f-4741-9878-51f6d2cacefb None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80462) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 10 08:22:25.969367 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 6/12] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:22:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 08:22:25.991930 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe1270e-eaaa-4a6b-b2a4-7f00f2a40ec0 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:26.005604 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-5a29415c-d13f-4741-9878-51f6d2cacefb None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:22:26.007174 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-5a29415c-d13f-4741-9878-51f6d2cacefb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:22:26.015235 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:26.096679 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe1270e-eaaa-4a6b-b2a4-7f00f2a40ec0 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:26.107878 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 7/13] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:22:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 08:22:26.129073 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a29415c-d13f-4741-9878-51f6d2cacefb None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:26.130371 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2ee243-2a4f-4174-9b54-2e603758f999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:26.131010 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2ee243-2a4f-4174-9b54-2e603758f999 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:26.131294 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2ee243-2a4f-4174-9b54-2e603758f999 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:26.131571 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2ee243-2a4f-4174-9b54-2e603758f999 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:26.143067 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2ee243-2a4f-4174-9b54-2e603758f999 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 08:22:26.152119 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-ca2ee243-2a4f-4174-9b54-2e603758f999 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 08:22:26.152119 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:22:26.152119 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:22:26.152119 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:22:26.152119 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:22:26.152119 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:22:26.152119 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:22:26.152119 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:22:26.152119 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:22:26.152119 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:22:26.152119 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:22:26.152119 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:22:26.152119 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 08:22:26.152119 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 08:22:26.152119 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 08:22:26.152119 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 08:22:26.153257 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 08:22:26.153257 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 08:22:26.153257 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 08:22:26.153257 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 08:22:26.153257 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:22:26.153257 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:22:26.153257 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:22:26.153257 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:22:26.153257 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:22:26.153257 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:22:26.153257 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:22:26.153257 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:22:26.153257 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:22:26.153257 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:22:26.153257 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:22:26.153257 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:22:26.154329 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:22:26.154329 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:22:26.154329 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:22:26.154329 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:22:26.154329 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 08:22:26.154329 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 08:22:26.154329 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 08:22:26.154329 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 08:22:26.154329 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 08:22:26.154329 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 08:22:26.154329 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 08:22:26.154329 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:22:26.154329 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 7/14] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Aug 10 08:22:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 08:22:26.159105 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2ee243-2a4f-4174-9b54-2e603758f999 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:26.232470 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a29415c-d13f-4741-9878-51f6d2cacefb None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:26.234771 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:26.240382 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:26.258351 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2ee243-2a4f-4174-9b54-2e603758f999 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:26.268014 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2deda05-a9cb-4e10-9c2a-012bf300bacc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:26.268874 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2deda05-a9cb-4e10-9c2a-012bf300bacc None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:26.269188 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2deda05-a9cb-4e10-9c2a-012bf300bacc None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:26.269463 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2deda05-a9cb-4e10-9c2a-012bf300bacc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:26.291919 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ce81dc-5d38-44f4-aeea-45f59b6aa57c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:26.292524 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ce81dc-5d38-44f4-aeea-45f59b6aa57c None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:26.292807 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ce81dc-5d38-44f4-aeea-45f59b6aa57c None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:26.293111 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ce81dc-5d38-44f4-aeea-45f59b6aa57c None admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:26.293776 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ce81dc-5d38-44f4-aeea-45f59b6aa57c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:22:26.294083 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ce81dc-5d38-44f4-aeea-45f59b6aa57c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:26.295490 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ce81dc-5d38-44f4-aeea-45f59b6aa57c None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:26.307577 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 8/15] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Aug 10 08:22:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:22:26.313162 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ce81dc-5d38-44f4-aeea-45f59b6aa57c None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:26.339353 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2deda05-a9cb-4e10-9c2a-012bf300bacc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:26.340939 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2deda05-a9cb-4e10-9c2a-012bf300bacc None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:26.413262 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ce81dc-5d38-44f4-aeea-45f59b6aa57c None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:26.441928 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 8/16] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Aug 10 08:22:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 10 08:22:26.448829 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2deda05-a9cb-4e10-9c2a-012bf300bacc None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:26.452096 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:26.554810 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2deda05-a9cb-4e10-9c2a-012bf300bacc None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:26.559496 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77957675-4765-4b66-9e34-8d41da4cc361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:26.560118 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77957675-4765-4b66-9e34-8d41da4cc361 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:26.560394 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77957675-4765-4b66-9e34-8d41da4cc361 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:26.560660 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77957675-4765-4b66-9e34-8d41da4cc361 None admin] PATH_INFO: `/v3/users/a79bffd3ad024a2f938d1f77cfd728d5` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:26.561810 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77957675-4765-4b66-9e34-8d41da4cc361 None admin] RBAC: Authorizing `identity:get_user(user_id=a79bffd3ad024a2f938d1f77cfd728d5)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:26.563242 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77957675-4765-4b66-9e34-8d41da4cc361 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:26.564795 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 9/17] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Aug 10 08:22:26 2022] GET /identity/v3/users/a79bffd3ad024a2f938d1f77cfd728d5 => generated 247 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:22:26.570618 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77957675-4765-4b66-9e34-8d41da4cc361 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:26.585121 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:26.589368 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a6aad5-d98e-4109-a38a-98dfea9dd0e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:26.590220 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a6aad5-d98e-4109-a38a-98dfea9dd0e2 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:26.590503 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a6aad5-d98e-4109-a38a-98dfea9dd0e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:26.590770 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a6aad5-d98e-4109-a38a-98dfea9dd0e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:26.608616 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a6aad5-d98e-4109-a38a-98dfea9dd0e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:26.610110 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a6aad5-d98e-4109-a38a-98dfea9dd0e2 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:26.627912 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-46a6aad5-d98e-4109-a38a-98dfea9dd0e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:22:26.627912 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:22:26.627912 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:22:26.627912 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:22:26.627912 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:22:26.627912 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:22:26.627912 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:22:26.627912 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:22:26.627912 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:22:26.627912 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:22:26.627912 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:22:26.627912 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:22:26.627912 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 08:22:26.627912 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 08:22:26.627912 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 08:22:26.627912 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 08:22:26.628858 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:22:26.628858 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:22:26.628858 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:22:26.628858 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:22:26.628858 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:22:26.628858 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:22:26.628858 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:22:26.628858 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:22:26.628858 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:22:26.628858 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:22:26.628858 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:22:26.628858 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:22:26.628858 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:22:26.628858 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:22:26.628858 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:22:26.628858 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:22:26.628858 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 08:22:26.630048 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 08:22:26.630048 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:22:26.630048 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:22:26.630048 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 9/18] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:22:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 08:22:26.635512 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a6aad5-d98e-4109-a38a-98dfea9dd0e2 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:26.670560 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77957675-4765-4b66-9e34-8d41da4cc361 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:26.690301 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:26.703268 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e8f66c-9239-4f6f-86d7-cba96e7bc7a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:26.703892 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e8f66c-9239-4f6f-86d7-cba96e7bc7a2 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:26.704169 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e8f66c-9239-4f6f-86d7-cba96e7bc7a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:26.704439 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e8f66c-9239-4f6f-86d7-cba96e7bc7a2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:26.705559 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e8f66c-9239-4f6f-86d7-cba96e7bc7a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:26.706988 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e8f66c-9239-4f6f-86d7-cba96e7bc7a2 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:26.708562 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 10/19] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:22:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:22:26.715000 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e8f66c-9239-4f6f-86d7-cba96e7bc7a2 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:26.735508 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a6aad5-d98e-4109-a38a-98dfea9dd0e2 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:26.769234 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40f499e-f76a-4fee-bb2b-67f740056949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:26.769858 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40f499e-f76a-4fee-bb2b-67f740056949 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:26.770118 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40f499e-f76a-4fee-bb2b-67f740056949 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:26.770388 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40f499e-f76a-4fee-bb2b-67f740056949 None admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:26.771033 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40f499e-f76a-4fee-bb2b-67f740056949 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:22:26.771336 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40f499e-f76a-4fee-bb2b-67f740056949 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:26.772726 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40f499e-f76a-4fee-bb2b-67f740056949 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:26.790306 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 10/20] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Aug 10 08:22:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:22:26.793528 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:26.814637 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e8f66c-9239-4f6f-86d7-cba96e7bc7a2 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:26.851701 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4a901b-551e-4025-aa10-149982f76ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:26.852331 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4a901b-551e-4025-aa10-149982f76ad9 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:26.852605 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4a901b-551e-4025-aa10-149982f76ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:26.852874 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4a901b-551e-4025-aa10-149982f76ad9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:26.853643 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4a901b-551e-4025-aa10-149982f76ad9 None admin] RBAC: Adding query filter params (role.id=ec3751692d9c4ecf81b4a71c936c46be, scope.domain.id=default, user.id=a79bffd3ad024a2f938d1f77cfd728d5) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:22:26.853951 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4a901b-551e-4025-aa10-149982f76ad9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:26.855424 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4a901b-551e-4025-aa10-149982f76ad9 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:26.863047 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 11/21] 10.222.0.57 () {56 vars in 1322 bytes} [Wed Aug 10 08:22:26 2022] GET /identity/v3/role_assignments?user.id=a79bffd3ad024a2f938d1f77cfd728d5&scope.domain.id=default&role.id=ec3751692d9c4ecf81b4a71c936c46be => generated 233 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:22:29.230318 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833ba135-511f-4083-89f5-c875aab7092c None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:29.231357 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833ba135-511f-4083-89f5-c875aab7092c None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:29.231837 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833ba135-511f-4083-89f5-c875aab7092c None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:29.233435 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 11/22] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:22:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:22:29.239252 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0f28d9-a422-491c-ba94-89ad66c46cb0 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:29.240071 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0f28d9-a422-491c-ba94-89ad66c46cb0 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:29.240547 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0f28d9-a422-491c-ba94-89ad66c46cb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:29.275107 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-8f0f28d9-a422-491c-ba94-89ad66c46cb0 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:22:29.276660 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-8f0f28d9-a422-491c-ba94-89ad66c46cb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:22:29.353141 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a95b3c3-a09f-4aed-a522-77886f04b326 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:29.353756 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a95b3c3-a09f-4aed-a522-77886f04b326 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:29.354241 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a95b3c3-a09f-4aed-a522-77886f04b326 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:29.355777 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 12/23] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:22:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 08:22:29.359672 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 12/24] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:22:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 08:22:29.361167 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c40a9b-a98c-4c98-baa8-4c4c21395786 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:29.361652 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c40a9b-a98c-4c98-baa8-4c4c21395786 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:29.362115 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c40a9b-a98c-4c98-baa8-4c4c21395786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:29.400543 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0f28d9-a422-491c-ba94-89ad66c46cb0 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:29.404263 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-62c40a9b-a98c-4c98-baa8-4c4c21395786 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:22:29.405712 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-62c40a9b-a98c-4c98-baa8-4c4c21395786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:22:29.541420 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:29.587013 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0f28d9-a422-491c-ba94-89ad66c46cb0 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:29.603369 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 13/25] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:22:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 08:22:29.604655 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3cc407-1b61-4a86-8823-c4a5cfa46c17 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:29.604952 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3cc407-1b61-4a86-8823-c4a5cfa46c17 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:29.605230 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3cc407-1b61-4a86-8823-c4a5cfa46c17 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:29.606478 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 14/26] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:22:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:22:29.614870 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3dbb3a-6662-48ac-9ee0-a9bb4409af0e None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:29.615443 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3dbb3a-6662-48ac-9ee0-a9bb4409af0e None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:29.615909 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3dbb3a-6662-48ac-9ee0-a9bb4409af0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:29.636628 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa878794-fe24-499a-833c-b29a792297c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:29.637303 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa878794-fe24-499a-833c-b29a792297c2 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:29.637587 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa878794-fe24-499a-833c-b29a792297c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:29.637859 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa878794-fe24-499a-833c-b29a792297c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:29.651623 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa878794-fe24-499a-833c-b29a792297c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:29.656206 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa878794-fe24-499a-833c-b29a792297c2 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:29.657471 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-db3dbb3a-6662-48ac-9ee0-a9bb4409af0e None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:22:29.658950 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-db3dbb3a-6662-48ac-9ee0-a9bb4409af0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:22:29.688993 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-aa878794-fe24-499a-833c-b29a792297c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:22:29.688993 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:22:29.688993 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:22:29.688993 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:22:29.688993 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:22:29.688993 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:22:29.688993 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:22:29.688993 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:22:29.688993 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:22:29.688993 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:22:29.688993 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:22:29.688993 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:22:29.688993 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 08:22:29.688993 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 08:22:29.688993 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 08:22:29.688993 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 08:22:29.689933 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:22:29.689933 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:22:29.689933 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:22:29.689933 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:22:29.689933 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:22:29.689933 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:22:29.689933 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:22:29.689933 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:22:29.689933 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:22:29.689933 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:22:29.689933 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:22:29.689933 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:22:29.689933 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:22:29.689933 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:22:29.689933 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:22:29.689933 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:22:29.689933 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 08:22:29.690741 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 08:22:29.690741 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:22:29.690741 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:22:29.697407 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 13/27] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:22:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:22:29.699291 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa878794-fe24-499a-833c-b29a792297c2 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:29.754089 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:29.795009 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 15/28] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:22:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 08:22:29.796520 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3dbb3a-6662-48ac-9ee0-a9bb4409af0e None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:29.860940 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa878794-fe24-499a-833c-b29a792297c2 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:29.901287 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:29.907068 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3dbb3a-6662-48ac-9ee0-a9bb4409af0e None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:29.914746 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31430c01-c789-466a-8dd2-ce7245a698d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:29.915382 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31430c01-c789-466a-8dd2-ce7245a698d4 None admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:29.915666 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31430c01-c789-466a-8dd2-ce7245a698d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:29.915942 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31430c01-c789-466a-8dd2-ce7245a698d4 None admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:29.916625 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31430c01-c789-466a-8dd2-ce7245a698d4 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:29.918154 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31430c01-c789-466a-8dd2-ce7245a698d4 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:29.948581 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e22823-6806-47fa-811d-cf408ca11516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:29.949210 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e22823-6806-47fa-811d-cf408ca11516 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:29.949491 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e22823-6806-47fa-811d-cf408ca11516 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:29.949763 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e22823-6806-47fa-811d-cf408ca11516 None admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:29.950456 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e22823-6806-47fa-811d-cf408ca11516 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:22:29.950761 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e22823-6806-47fa-811d-cf408ca11516 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:29.952132 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e22823-6806-47fa-811d-cf408ca11516 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:29.969121 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 16/29] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Aug 10 08:22:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:22:29.969121 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e22823-6806-47fa-811d-cf408ca11516 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:30.113331 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:30.153733 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 14/30] 10.222.0.57 () {60 vars in 1131 bytes} [Wed Aug 10 08:22:29 2022] POST /identity/v3/roles => generated 219 bytes in 455 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:22:30.155449 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8311152b-6127-455b-a269-38e1afb93645 None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:30.155914 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8311152b-6127-455b-a269-38e1afb93645 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:30.156351 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8311152b-6127-455b-a269-38e1afb93645 None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:30.162498 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 15/31] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:22:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:22:30.162498 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8311152b-6127-455b-a269-38e1afb93645 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:30.200528 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e22823-6806-47fa-811d-cf408ca11516 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:30.241551 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ba596f-ff32-47b1-be21-f4e5c589bd91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:30.242540 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ba596f-ff32-47b1-be21-f4e5c589bd91 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:30.242710 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ba596f-ff32-47b1-be21-f4e5c589bd91 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:30.243137 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ba596f-ff32-47b1-be21-f4e5c589bd91 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:30.256757 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ba596f-ff32-47b1-be21-f4e5c589bd91 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 08:22:30.263952 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-c3ba596f-ff32-47b1-be21-f4e5c589bd91 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 08:22:30.263952 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:22:30.263952 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:22:30.263952 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:22:30.263952 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:22:30.263952 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:22:30.263952 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:22:30.263952 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:22:30.263952 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:22:30.263952 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:22:30.263952 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:22:30.263952 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:22:30.263952 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 08:22:30.263952 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 08:22:30.263952 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 08:22:30.263952 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 08:22:30.265118 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 08:22:30.265118 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 08:22:30.265118 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 08:22:30.265118 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 08:22:30.265118 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:22:30.265118 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:22:30.265118 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:22:30.265118 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:22:30.265118 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:22:30.265118 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:22:30.265118 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:22:30.265118 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:22:30.265118 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:22:30.265118 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:22:30.265118 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:22:30.265118 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:22:30.266127 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:22:30.266127 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:22:30.266127 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:22:30.266127 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:22:30.266127 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 08:22:30.266127 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 08:22:30.266127 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 08:22:30.266127 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 08:22:30.266127 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 08:22:30.266127 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 08:22:30.266127 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 08:22:30.266127 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:22:30.267549 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 17/32] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Aug 10 08:22:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:22:30.268734 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64df829-44fe-44dd-afd9-eb728a0409f2 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:30.273929 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64df829-44fe-44dd-afd9-eb728a0409f2 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:30.273929 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64df829-44fe-44dd-afd9-eb728a0409f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:30.296907 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-e64df829-44fe-44dd-afd9-eb728a0409f2 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:22:30.299230 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-e64df829-44fe-44dd-afd9-eb728a0409f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:22:30.302705 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8311152b-6127-455b-a269-38e1afb93645 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:30.342944 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddffe983-a699-4147-a206-956cea6cc56f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:30.343865 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddffe983-a699-4147-a206-956cea6cc56f None admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:30.344223 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddffe983-a699-4147-a206-956cea6cc56f None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:30.344638 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddffe983-a699-4147-a206-956cea6cc56f None admin] PATH_INFO: `/v3/projects` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:30.345431 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddffe983-a699-4147-a206-956cea6cc56f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:30.347169 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddffe983-a699-4147-a206-956cea6cc56f None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:30.395997 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 18/33] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:22:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 08:22:30.401325 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64df829-44fe-44dd-afd9-eb728a0409f2 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:30.443382 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:30.497325 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 16/34] 10.222.0.57 () {60 vars in 1141 bytes} [Wed Aug 10 08:22:30 2022] POST /identity/v3/projects => generated 301 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:22:30.498512 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddffe983-a699-4147-a206-956cea6cc56f None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:30.527489 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64df829-44fe-44dd-afd9-eb728a0409f2 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:30.572843 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:30.580873 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2022dab0-1a5b-49e8-b81d-5fc0c21ff008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:30.581534 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2022dab0-1a5b-49e8-b81d-5fc0c21ff008 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:30.581869 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2022dab0-1a5b-49e8-b81d-5fc0c21ff008 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:30.582145 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2022dab0-1a5b-49e8-b81d-5fc0c21ff008 None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:30.582878 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2022dab0-1a5b-49e8-b81d-5fc0c21ff008 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:22:30.583189 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2022dab0-1a5b-49e8-b81d-5fc0c21ff008 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:30.584961 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2022dab0-1a5b-49e8-b81d-5fc0c21ff008 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:30.603219 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 19/35] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Aug 10 08:22:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:22:30.609441 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8fa4e2-9015-4675-a896-76314bba0bca None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:30.609747 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8fa4e2-9015-4675-a896-76314bba0bca None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:30.610017 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8fa4e2-9015-4675-a896-76314bba0bca None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:30.611736 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 20/36] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:22:30 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:22:30.613091 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8fa4e2-9015-4675-a896-76314bba0bca None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:30.668171 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddffe983-a699-4147-a206-956cea6cc56f None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:30.709898 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1765e8-516b-4c32-a319-36412a892782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:30.711158 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1765e8-516b-4c32-a319-36412a892782 None admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:30.711572 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1765e8-516b-4c32-a319-36412a892782 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:30.712658 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1765e8-516b-4c32-a319-36412a892782 None admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:30.713541 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1765e8-516b-4c32-a319-36412a892782 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:30.714800 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1765e8-516b-4c32-a319-36412a892782 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:30.748357 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:30.763107 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8fa4e2-9015-4675-a896-76314bba0bca None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:30.790156 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 17/37] 10.222.0.57 () {60 vars in 1131 bytes} [Wed Aug 10 08:22:30 2022] POST /identity/v3/roles => generated 225 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:22:30.791726 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed34593-f7fc-4130-b945-ece20f0a319b None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:30.792080 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed34593-f7fc-4130-b945-ece20f0a319b None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:30.792415 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed34593-f7fc-4130-b945-ece20f0a319b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:30.808174 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57145e94-80e6-49bf-982b-d1f69d055c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:30.809583 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57145e94-80e6-49bf-982b-d1f69d055c1d None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:30.810137 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57145e94-80e6-49bf-982b-d1f69d055c1d None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:30.810632 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57145e94-80e6-49bf-982b-d1f69d055c1d None admin] PATH_INFO: `/v3/users/a79bffd3ad024a2f938d1f77cfd728d5` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:30.812807 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57145e94-80e6-49bf-982b-d1f69d055c1d None admin] RBAC: Authorizing `identity:get_user(user_id=a79bffd3ad024a2f938d1f77cfd728d5)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:30.813829 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-5ed34593-f7fc-4130-b945-ece20f0a319b None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:22:30.814966 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57145e94-80e6-49bf-982b-d1f69d055c1d None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:30.815458 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-5ed34593-f7fc-4130-b945-ece20f0a319b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:22:30.816711 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 21/38] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Aug 10 08:22:30 2022] GET /identity/v3/users/a79bffd3ad024a2f938d1f77cfd728d5 => generated 247 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:22:30.818290 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2131a0-a2b8-41bf-8611-c4529c62b3d8 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:30.818563 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2131a0-a2b8-41bf-8611-c4529c62b3d8 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:30.819806 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2131a0-a2b8-41bf-8611-c4529c62b3d8 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:30.821779 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 22/39] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:22:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:22:30.823273 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2131a0-a2b8-41bf-8611-c4529c62b3d8 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:30.897848 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 18/40] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:22:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 08:22:30.900313 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddbd624-42a1-4355-b1af-2eed8026972e None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:30.900609 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddbd624-42a1-4355-b1af-2eed8026972e None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:30.900889 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddbd624-42a1-4355-b1af-2eed8026972e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:30.925646 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-3ddbd624-42a1-4355-b1af-2eed8026972e None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:22:30.926839 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-3ddbd624-42a1-4355-b1af-2eed8026972e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:22:30.942352 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2131a0-a2b8-41bf-8611-c4529c62b3d8 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:30.959376 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:30.983933 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996e65db-3548-4724-a55c-7237e5087aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:30.984154 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996e65db-3548-4724-a55c-7237e5087aa6 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:30.984365 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996e65db-3548-4724-a55c-7237e5087aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:30.984607 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996e65db-3548-4724-a55c-7237e5087aa6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:30.985762 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996e65db-3548-4724-a55c-7237e5087aa6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:30.987201 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996e65db-3548-4724-a55c-7237e5087aa6 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:30.988968 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 23/41] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:22:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:22:30.990204 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996e65db-3548-4724-a55c-7237e5087aa6 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:31.010431 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 19/42] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:22:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 08:22:31.013927 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddbd624-42a1-4355-b1af-2eed8026972e None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:31.018727 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:31.097513 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996e65db-3548-4724-a55c-7237e5087aa6 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:31.113431 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddbd624-42a1-4355-b1af-2eed8026972e None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:31.132014 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba604ba-90c7-4b78-a639-0f828aed9e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:31.132720 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba604ba-90c7-4b78-a639-0f828aed9e39 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:31.133037 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba604ba-90c7-4b78-a639-0f828aed9e39 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:31.133309 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba604ba-90c7-4b78-a639-0f828aed9e39 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:31.134430 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba604ba-90c7-4b78-a639-0f828aed9e39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:31.135894 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba604ba-90c7-4b78-a639-0f828aed9e39 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:31.137489 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 24/43] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:22:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:22:31.138229 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba604ba-90c7-4b78-a639-0f828aed9e39 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:31.152217 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442d45f5-8d06-4839-8e39-146d78c7fa27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:31.152885 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442d45f5-8d06-4839-8e39-146d78c7fa27 None admin] REQUEST_METHOD: `PUT` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:31.153193 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442d45f5-8d06-4839-8e39-146d78c7fa27 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:31.153475 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442d45f5-8d06-4839-8e39-146d78c7fa27 None admin] PATH_INFO: `/v3/domains/default/users/a79bffd3ad024a2f938d1f77cfd728d5/roles/ec3751692d9c4ecf81b4a71c936c46be` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:31.155440 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442d45f5-8d06-4839-8e39-146d78c7fa27 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=a79bffd3ad024a2f938d1f77cfd728d5, role_id=ec3751692d9c4ecf81b4a71c936c46be)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:31.156888 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442d45f5-8d06-4839-8e39-146d78c7fa27 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:31.227337 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:31.233892 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 20/44] 10.222.0.57 () {58 vars in 1361 bytes} [Wed Aug 10 08:22:31 2022] PUT /identity/v3/domains/default/users/a79bffd3ad024a2f938d1f77cfd728d5/roles/ec3751692d9c4ecf81b4a71c936c46be => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:22:31.234705 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442d45f5-8d06-4839-8e39-146d78c7fa27 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:31.245179 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba604ba-90c7-4b78-a639-0f828aed9e39 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:31.257396 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:31.283439 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828d608a-dc97-4b56-ae71-1bc90056278a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:31.284323 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828d608a-dc97-4b56-ae71-1bc90056278a None admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:31.284742 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828d608a-dc97-4b56-ae71-1bc90056278a None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:31.285186 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828d608a-dc97-4b56-ae71-1bc90056278a None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:31.286086 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828d608a-dc97-4b56-ae71-1bc90056278a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:31.287686 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828d608a-dc97-4b56-ae71-1bc90056278a None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:31.349821 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442d45f5-8d06-4839-8e39-146d78c7fa27 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:31.391025 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca344fc4-2baf-4610-aeb2-e7a692efb53a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:31.391690 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca344fc4-2baf-4610-aeb2-e7a692efb53a None admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:31.391928 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca344fc4-2baf-4610-aeb2-e7a692efb53a None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:31.392189 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca344fc4-2baf-4610-aeb2-e7a692efb53a None admin] PATH_INFO: `/v3/projects` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:31.392835 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca344fc4-2baf-4610-aeb2-e7a692efb53a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:31.394320 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca344fc4-2baf-4610-aeb2-e7a692efb53a None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:31.480231 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 21/45] 10.222.0.57 () {60 vars in 1141 bytes} [Wed Aug 10 08:22:31 2022] POST /identity/v3/projects => generated 312 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:22:31.485500 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 25/46] 10.222.0.57 () {60 vars in 1131 bytes} [Wed Aug 10 08:22:31 2022] POST /identity/v3/roles => generated 223 bytes in 347 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:22:33.284179 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033cd243-5db9-4eda-bc52-bc76b4273bfb None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:33.284701 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033cd243-5db9-4eda-bc52-bc76b4273bfb None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:33.284701 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033cd243-5db9-4eda-bc52-bc76b4273bfb None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:33.286080 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 22/47] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:22:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:22:33.291877 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126298a6-db3a-4aa2-9b2c-16597e313922 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:33.292284 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126298a6-db3a-4aa2-9b2c-16597e313922 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:33.292435 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126298a6-db3a-4aa2-9b2c-16597e313922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:33.311241 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-126298a6-db3a-4aa2-9b2c-16597e313922 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:22:33.312757 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-126298a6-db3a-4aa2-9b2c-16597e313922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:22:33.379102 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 26/48] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:22:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 08:22:33.403245 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033cd243-5db9-4eda-bc52-bc76b4273bfb None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:33.473904 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:33.504657 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033cd243-5db9-4eda-bc52-bc76b4273bfb None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:33.531485 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5794d34-9141-4b1c-941e-98117cd453dd None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:33.531868 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5794d34-9141-4b1c-941e-98117cd453dd None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:33.532237 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5794d34-9141-4b1c-941e-98117cd453dd None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:33.533725 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 27/49] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:22:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 08:22:33.538547 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ceae97-d166-49ab-895d-ec3cf6e52680 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:33.538676 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3858e4ef-493a-4685-b388-14ec7fd266de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:33.538829 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ceae97-d166-49ab-895d-ec3cf6e52680 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:33.539028 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ceae97-d166-49ab-895d-ec3cf6e52680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:33.539259 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3858e4ef-493a-4685-b388-14ec7fd266de None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:33.539545 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3858e4ef-493a-4685-b388-14ec7fd266de None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:33.539822 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3858e4ef-493a-4685-b388-14ec7fd266de None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:33.550014 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3858e4ef-493a-4685-b388-14ec7fd266de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 08:22:33.556656 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-3858e4ef-493a-4685-b388-14ec7fd266de None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 08:22:33.556656 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:22:33.556656 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:22:33.556656 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:22:33.556656 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:22:33.556656 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:22:33.556656 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:22:33.556656 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:22:33.556656 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:22:33.556656 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:22:33.556656 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:22:33.556656 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:22:33.556656 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 08:22:33.556656 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 08:22:33.556656 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 08:22:33.556656 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 08:22:33.557558 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 08:22:33.557558 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 08:22:33.557558 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 08:22:33.557558 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 08:22:33.557558 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:22:33.557558 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:22:33.557558 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:22:33.557558 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:22:33.557558 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:22:33.557558 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:22:33.557558 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:22:33.557558 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:22:33.557558 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:22:33.557558 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:22:33.557558 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:22:33.557558 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:22:33.558322 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:22:33.558322 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:22:33.558322 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:22:33.558322 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:22:33.558322 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 08:22:33.558322 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 08:22:33.558322 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 08:22:33.558322 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 08:22:33.558322 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 08:22:33.558322 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 08:22:33.558322 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 08:22:33.558322 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:22:33.558941 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-21ceae97-d166-49ab-895d-ec3cf6e52680 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:22:33.559020 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 23/50] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Aug 10 08:22:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:22:33.559095 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-21ceae97-d166-49ab-895d-ec3cf6e52680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:22:33.565309 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3858e4ef-493a-4685-b388-14ec7fd266de None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:33.624467 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 28/51] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:22:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 08:22:33.647666 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ceae97-d166-49ab-895d-ec3cf6e52680 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:33.661731 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:33.663429 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3858e4ef-493a-4685-b388-14ec7fd266de None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:33.691339 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:33.697739 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf90f2c2-d2e6-404c-8f1f-de48759e662d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:33.698397 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf90f2c2-d2e6-404c-8f1f-de48759e662d None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:33.698726 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf90f2c2-d2e6-404c-8f1f-de48759e662d None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:33.698998 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf90f2c2-d2e6-404c-8f1f-de48759e662d None admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:33.699667 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf90f2c2-d2e6-404c-8f1f-de48759e662d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:22:33.699973 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf90f2c2-d2e6-404c-8f1f-de48759e662d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:33.701411 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf90f2c2-d2e6-404c-8f1f-de48759e662d None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:33.709825 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 24/52] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Aug 10 08:22:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:22:33.715520 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf90f2c2-d2e6-404c-8f1f-de48759e662d None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:33.746923 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ceae97-d166-49ab-895d-ec3cf6e52680 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:33.780359 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bd6522-59e1-46b8-8d50-8eb2a47da205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:33.781013 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bd6522-59e1-46b8-8d50-8eb2a47da205 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:33.781301 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bd6522-59e1-46b8-8d50-8eb2a47da205 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:33.781579 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bd6522-59e1-46b8-8d50-8eb2a47da205 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:33.782685 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bd6522-59e1-46b8-8d50-8eb2a47da205 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:33.784267 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bd6522-59e1-46b8-8d50-8eb2a47da205 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:33.786006 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 29/53] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:22:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:22:33.791754 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bd6522-59e1-46b8-8d50-8eb2a47da205 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:33.815827 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf90f2c2-d2e6-404c-8f1f-de48759e662d None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:33.848541 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df19c262-ac11-4bf5-acf0-7ec3c1ae575f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:33.849197 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df19c262-ac11-4bf5-acf0-7ec3c1ae575f None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:33.849481 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df19c262-ac11-4bf5-acf0-7ec3c1ae575f None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:33.849752 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df19c262-ac11-4bf5-acf0-7ec3c1ae575f None admin] PATH_INFO: `/v3/users/a79bffd3ad024a2f938d1f77cfd728d5` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:33.850895 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df19c262-ac11-4bf5-acf0-7ec3c1ae575f None admin] RBAC: Authorizing `identity:get_user(user_id=a79bffd3ad024a2f938d1f77cfd728d5)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:33.852481 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df19c262-ac11-4bf5-acf0-7ec3c1ae575f None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:33.854083 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 25/54] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Aug 10 08:22:33 2022] GET /identity/v3/users/a79bffd3ad024a2f938d1f77cfd728d5 => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:22:33.859327 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df19c262-ac11-4bf5-acf0-7ec3c1ae575f None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:33.877234 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:33.896360 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bd6522-59e1-46b8-8d50-8eb2a47da205 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:33.919809 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:33.931427 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2f2c3d-d063-4b95-bb5f-7e9178be8430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:33.932043 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2f2c3d-d063-4b95-bb5f-7e9178be8430 None admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:33.932330 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2f2c3d-d063-4b95-bb5f-7e9178be8430 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:33.932607 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2f2c3d-d063-4b95-bb5f-7e9178be8430 None admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:33.933301 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2f2c3d-d063-4b95-bb5f-7e9178be8430 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:33.934894 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2f2c3d-d063-4b95-bb5f-7e9178be8430 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:33.961497 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df19c262-ac11-4bf5-acf0-7ec3c1ae575f None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:33.994734 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbfc3eb-28c3-4f9b-8127-8d9ea2ad385b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:33.995353 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbfc3eb-28c3-4f9b-8127-8d9ea2ad385b None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:33.995630 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbfc3eb-28c3-4f9b-8127-8d9ea2ad385b None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:33.995899 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbfc3eb-28c3-4f9b-8127-8d9ea2ad385b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:33.997296 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbfc3eb-28c3-4f9b-8127-8d9ea2ad385b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:33.998742 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbfc3eb-28c3-4f9b-8127-8d9ea2ad385b None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:34.000284 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 26/55] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:22:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:22:34.005932 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbfc3eb-28c3-4f9b-8127-8d9ea2ad385b None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:34.036981 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 30/56] 10.222.0.57 () {60 vars in 1140 bytes} [Wed Aug 10 08:22:33 2022] POST /identity/v3/projects => generated 298 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 08:22:34.107810 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbfc3eb-28c3-4f9b-8127-8d9ea2ad385b None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:34.135958 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:34.143557 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5f167a-a3c8-428b-affb-a18af41b2bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:34.144181 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5f167a-a3c8-428b-affb-a18af41b2bd0 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:34.144452 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5f167a-a3c8-428b-affb-a18af41b2bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:34.144717 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5f167a-a3c8-428b-affb-a18af41b2bd0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:34.145486 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5f167a-a3c8-428b-affb-a18af41b2bd0 None admin] RBAC: Adding query filter params (role.id=ec3751692d9c4ecf81b4a71c936c46be, scope.domain.id=default, user.id=a79bffd3ad024a2f938d1f77cfd728d5) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:22:34.145789 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5f167a-a3c8-428b-affb-a18af41b2bd0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:34.147269 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5f167a-a3c8-428b-affb-a18af41b2bd0 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:34.153838 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 27/57] 10.222.0.57 () {56 vars in 1322 bytes} [Wed Aug 10 08:22:34 2022] GET /identity/v3/role_assignments?user.id=a79bffd3ad024a2f938d1f77cfd728d5&scope.domain.id=default&role.id=ec3751692d9c4ecf81b4a71c936c46be => generated 530 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:22:36.067637 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34d8438-97b2-4927-87c0-ea21e868b263 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:36.068183 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34d8438-97b2-4927-87c0-ea21e868b263 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:36.068183 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34d8438-97b2-4927-87c0-ea21e868b263 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:36.069550 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 31/58] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:22:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:22:36.074720 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a80fe7-f312-4369-b826-fc599466f311 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:36.075108 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a80fe7-f312-4369-b826-fc599466f311 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:36.075275 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a80fe7-f312-4369-b826-fc599466f311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:36.093303 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-e6a80fe7-f312-4369-b826-fc599466f311 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:22:36.094472 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-e6a80fe7-f312-4369-b826-fc599466f311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:22:36.159122 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 28/59] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:22:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 08:22:36.183278 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34d8438-97b2-4927-87c0-ea21e868b263 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:36.280809 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34d8438-97b2-4927-87c0-ea21e868b263 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:36.309618 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:36.314021 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756426c6-abb4-426c-9b4a-9477eb51fb2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:36.314627 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756426c6-abb4-426c-9b4a-9477eb51fb2b None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:36.314906 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756426c6-abb4-426c-9b4a-9477eb51fb2b None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:36.315171 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756426c6-abb4-426c-9b4a-9477eb51fb2b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:36.316269 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756426c6-abb4-426c-9b4a-9477eb51fb2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:36.317781 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756426c6-abb4-426c-9b4a-9477eb51fb2b None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:36.319266 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 32/60] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:22:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:22:36.324737 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a80fe7-f312-4369-b826-fc599466f311 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:36.425831 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a80fe7-f312-4369-b826-fc599466f311 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:36.447973 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:36.459275 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad07f1f1-f0ee-4ef6-a7b7-988dee5b9215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:36.459886 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad07f1f1-f0ee-4ef6-a7b7-988dee5b9215 None admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:36.460167 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad07f1f1-f0ee-4ef6-a7b7-988dee5b9215 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:36.460641 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad07f1f1-f0ee-4ef6-a7b7-988dee5b9215 None admin] PATH_INFO: `/v3/users` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:36.461320 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad07f1f1-f0ee-4ef6-a7b7-988dee5b9215 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:36.462869 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad07f1f1-f0ee-4ef6-a7b7-988dee5b9215 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:36.593825 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 29/61] 10.222.0.57 () {60 vars in 1132 bytes} [Wed Aug 10 08:22:36 2022] POST /identity/v3/users => generated 275 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:22:39.705175 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be12966-180b-422a-9eb9-e387ce1dd14e None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:39.706195 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be12966-180b-422a-9eb9-e387ce1dd14e None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:39.706684 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be12966-180b-422a-9eb9-e387ce1dd14e None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:39.708444 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 33/62] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:22:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:22:39.715127 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7011d266-a071-4dc4-9704-6b70add3e595 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:39.715956 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7011d266-a071-4dc4-9704-6b70add3e595 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:39.716437 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7011d266-a071-4dc4-9704-6b70add3e595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:39.746407 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-7011d266-a071-4dc4-9704-6b70add3e595 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:22:39.747945 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-7011d266-a071-4dc4-9704-6b70add3e595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:22:39.838735 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 30/63] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:22:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 08:22:39.886454 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be12966-180b-422a-9eb9-e387ce1dd14e None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:39.896532 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0598a2ba-cad5-465f-81e4-d53af6ed6b8f None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:39.900022 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0598a2ba-cad5-465f-81e4-d53af6ed6b8f None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:39.900508 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0598a2ba-cad5-465f-81e4-d53af6ed6b8f None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:39.902123 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 31/64] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:22:39 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:22:39.908540 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2b2a18-644a-4d59-8430-2a8ec86b9b66 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:39.908827 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2b2a18-644a-4d59-8430-2a8ec86b9b66 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:39.909138 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2b2a18-644a-4d59-8430-2a8ec86b9b66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:39.938661 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-cf2b2a18-644a-4d59-8430-2a8ec86b9b66 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:22:39.940758 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-cf2b2a18-644a-4d59-8430-2a8ec86b9b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:22:40.069112 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 32/65] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:22:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 08:22:40.071010 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f794e9d-fe8a-4a77-a91b-823de1101c52 None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:40.071300 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f794e9d-fe8a-4a77-a91b-823de1101c52 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:40.071577 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f794e9d-fe8a-4a77-a91b-823de1101c52 None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:40.072871 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 33/66] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:22:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:22:40.073314 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be12966-180b-422a-9eb9-e387ce1dd14e None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:40.087146 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00855427-8fe3-4dff-a433-b4ce3af430ae None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:40.087432 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00855427-8fe3-4dff-a433-b4ce3af430ae None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:40.087705 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00855427-8fe3-4dff-a433-b4ce3af430ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:40.113333 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-00855427-8fe3-4dff-a433-b4ce3af430ae None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:22:40.114529 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-00855427-8fe3-4dff-a433-b4ce3af430ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:22:40.118643 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe1abcc-c990-47e3-9680-e3fa4c8eb47a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:40.119492 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe1abcc-c990-47e3-9680-e3fa4c8eb47a None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:40.119950 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe1abcc-c990-47e3-9680-e3fa4c8eb47a None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:40.120397 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe1abcc-c990-47e3-9680-e3fa4c8eb47a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:40.136706 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe1abcc-c990-47e3-9680-e3fa4c8eb47a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 08:22:40.144006 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-0fe1abcc-c990-47e3-9680-e3fa4c8eb47a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 08:22:40.144006 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:22:40.144006 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:22:40.144006 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:22:40.144006 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:22:40.144006 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:22:40.144006 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:22:40.144006 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:22:40.144006 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:22:40.144006 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:22:40.144006 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:22:40.144006 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:22:40.144006 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 08:22:40.144006 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 08:22:40.144006 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 08:22:40.144006 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 08:22:40.144902 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 08:22:40.144902 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 08:22:40.144902 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 08:22:40.144902 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 08:22:40.144902 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:22:40.144902 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:22:40.144902 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:22:40.144902 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:22:40.144902 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:22:40.144902 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:22:40.144902 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:22:40.144902 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:22:40.144902 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:22:40.144902 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:22:40.144902 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:22:40.144902 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:22:40.145724 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:22:40.145724 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:22:40.145724 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:22:40.145724 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:22:40.145724 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 08:22:40.145724 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 08:22:40.145724 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 08:22:40.145724 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 08:22:40.145724 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 08:22:40.145724 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 08:22:40.145724 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 08:22:40.145724 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:22:40.146320 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 34/67] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Aug 10 08:22:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:22:40.147491 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe1abcc-c990-47e3-9680-e3fa4c8eb47a None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:40.158366 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:40.213200 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 34/68] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:22:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 08:22:40.214435 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00855427-8fe3-4dff-a433-b4ce3af430ae None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:40.222670 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:40.276014 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe1abcc-c990-47e3-9680-e3fa4c8eb47a None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:40.311255 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182953dd-232f-4fab-9804-632b22600c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:40.311914 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182953dd-232f-4fab-9804-632b22600c54 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:40.312198 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182953dd-232f-4fab-9804-632b22600c54 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:40.312473 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182953dd-232f-4fab-9804-632b22600c54 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:40.322998 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182953dd-232f-4fab-9804-632b22600c54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 08:22:40.328083 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-182953dd-232f-4fab-9804-632b22600c54 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 08:22:40.328083 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:22:40.328083 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:22:40.328083 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:22:40.328083 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:22:40.328083 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:22:40.328083 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:22:40.328083 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:22:40.328083 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:22:40.328083 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:22:40.328083 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:22:40.328083 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:22:40.328083 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 08:22:40.328083 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 08:22:40.328083 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 08:22:40.328083 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 08:22:40.333738 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 08:22:40.333738 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 08:22:40.333738 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 08:22:40.333738 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 08:22:40.333738 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:22:40.333738 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:22:40.333738 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:22:40.333738 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:22:40.333738 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:22:40.333738 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:22:40.333738 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:22:40.333738 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:22:40.333738 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:22:40.333738 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:22:40.333738 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:22:40.333738 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:22:40.334522 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:22:40.334522 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:22:40.334522 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:22:40.334522 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:22:40.334522 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 08:22:40.334522 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 08:22:40.334522 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 08:22:40.334522 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 08:22:40.334522 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 08:22:40.334522 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 08:22:40.334522 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 08:22:40.334522 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:22:40.334522 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 35/69] 10.222.0.57 () {56 vars in 1114 bytes} [Wed Aug 10 08:22:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:22:40.334522 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab041d3-5b7a-43d2-839c-562bf0eeff7c None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:40.335372 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab041d3-5b7a-43d2-839c-562bf0eeff7c None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:40.335492 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00855427-8fe3-4dff-a433-b4ce3af430ae None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:40.335565 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab041d3-5b7a-43d2-839c-562bf0eeff7c None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:40.335565 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 36/70] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:22:40 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:22:40.335565 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab041d3-5b7a-43d2-839c-562bf0eeff7c None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:40.369842 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deffc42d-e225-4a49-87b4-a053af49e6f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:40.370447 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deffc42d-e225-4a49-87b4-a053af49e6f9 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:40.370731 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deffc42d-e225-4a49-87b4-a053af49e6f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:40.371005 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deffc42d-e225-4a49-87b4-a053af49e6f9 None admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:40.371927 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deffc42d-e225-4a49-87b4-a053af49e6f9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:22:40.372230 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deffc42d-e225-4a49-87b4-a053af49e6f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:40.373699 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deffc42d-e225-4a49-87b4-a053af49e6f9 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:40.382058 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 35/71] 10.222.0.57 () {56 vars in 1101 bytes} [Wed Aug 10 08:22:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:22:40.384390 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7e1118-0d7d-4e03-8703-f49db3dbd580 None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:40.384684 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7e1118-0d7d-4e03-8703-f49db3dbd580 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:40.384979 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7e1118-0d7d-4e03-8703-f49db3dbd580 None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:40.386377 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 36/72] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:22:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:22:40.387141 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7e1118-0d7d-4e03-8703-f49db3dbd580 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:40.420278 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:40.445778 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab041d3-5b7a-43d2-839c-562bf0eeff7c None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:40.465006 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:40.478765 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4491aaa-592c-4ed2-8ebd-1505832bbc33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:40.479393 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4491aaa-592c-4ed2-8ebd-1505832bbc33 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:40.479673 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4491aaa-592c-4ed2-8ebd-1505832bbc33 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:40.479948 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4491aaa-592c-4ed2-8ebd-1505832bbc33 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:40.490157 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4491aaa-592c-4ed2-8ebd-1505832bbc33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 08:22:40.493870 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7e1118-0d7d-4e03-8703-f49db3dbd580 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:40.496008 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-f4491aaa-592c-4ed2-8ebd-1505832bbc33 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 08:22:40.496008 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:22:40.496008 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:22:40.496008 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:22:40.496008 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:22:40.496008 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:22:40.496008 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:22:40.496008 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:22:40.496008 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:22:40.496008 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:22:40.496008 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:22:40.496008 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:22:40.496008 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 08:22:40.496008 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 08:22:40.496008 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 08:22:40.496008 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 08:22:40.497172 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 08:22:40.497172 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 08:22:40.497172 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 08:22:40.497172 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 08:22:40.497172 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:22:40.497172 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:22:40.497172 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:22:40.497172 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:22:40.497172 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:22:40.497172 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:22:40.497172 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:22:40.497172 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:22:40.497172 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:22:40.497172 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:22:40.497172 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:22:40.497172 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:22:40.497934 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:22:40.497934 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:22:40.497934 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:22:40.497934 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:22:40.497934 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 08:22:40.497934 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 08:22:40.497934 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 08:22:40.497934 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 08:22:40.497934 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 08:22:40.497934 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 08:22:40.497934 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 08:22:40.497934 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:22:40.497934 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 37/73] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Aug 10 08:22:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:22:40.499036 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f68b597-c45d-4d12-8665-44e770d1c9c5 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:40.499036 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f68b597-c45d-4d12-8665-44e770d1c9c5 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:40.499311 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f68b597-c45d-4d12-8665-44e770d1c9c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:40.517340 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-4f68b597-c45d-4d12-8665-44e770d1c9c5 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:22:40.518621 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-4f68b597-c45d-4d12-8665-44e770d1c9c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:22:40.530018 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffa33a8-bb87-448a-a41f-2b422c69fb2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:40.530629 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffa33a8-bb87-448a-a41f-2b422c69fb2e None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:40.530912 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffa33a8-bb87-448a-a41f-2b422c69fb2e None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:40.531186 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffa33a8-bb87-448a-a41f-2b422c69fb2e None admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:40.531848 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffa33a8-bb87-448a-a41f-2b422c69fb2e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:22:40.532151 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffa33a8-bb87-448a-a41f-2b422c69fb2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:40.533592 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffa33a8-bb87-448a-a41f-2b422c69fb2e None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:40.541570 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 37/74] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Aug 10 08:22:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:22:40.542359 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffa33a8-bb87-448a-a41f-2b422c69fb2e None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:40.586220 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 38/75] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:22:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 08:22:40.587224 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d736a91-d881-483e-b990-ed06418ded88 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:40.587521 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d736a91-d881-483e-b990-ed06418ded88 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:40.587774 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d736a91-d881-483e-b990-ed06418ded88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:40.613879 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-1d736a91-d881-483e-b990-ed06418ded88 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:22:40.615051 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-1d736a91-d881-483e-b990-ed06418ded88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:22:40.634247 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:40.642933 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffa33a8-bb87-448a-a41f-2b422c69fb2e None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:40.675348 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fea09ff-2f65-4c6a-a568-2c791bc6a655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:40.675979 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fea09ff-2f65-4c6a-a568-2c791bc6a655 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:40.676258 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fea09ff-2f65-4c6a-a568-2c791bc6a655 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:40.676533 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fea09ff-2f65-4c6a-a568-2c791bc6a655 None admin] PATH_INFO: `/v3/users/8179b7e759db449e9295e696b2333c9c` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:40.682778 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 39/76] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:22:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 08:22:40.683606 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d736a91-d881-483e-b990-ed06418ded88 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:40.690560 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fea09ff-2f65-4c6a-a568-2c791bc6a655 None admin] RBAC: Authorizing `identity:get_user(user_id=8179b7e759db449e9295e696b2333c9c)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:40.692018 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fea09ff-2f65-4c6a-a568-2c791bc6a655 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:40.693664 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 38/77] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Aug 10 08:22:40 2022] GET /identity/v3/users/8179b7e759db449e9295e696b2333c9c => generated 275 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:22:40.694427 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fea09ff-2f65-4c6a-a568-2c791bc6a655 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:40.735469 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:40.791522 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fea09ff-2f65-4c6a-a568-2c791bc6a655 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:40.792148 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d736a91-d881-483e-b990-ed06418ded88 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:40.825158 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591c8f97-a046-4590-a73f-8ab2f5196422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:40.825790 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591c8f97-a046-4590-a73f-8ab2f5196422 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:40.826074 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591c8f97-a046-4590-a73f-8ab2f5196422 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:40.826348 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591c8f97-a046-4590-a73f-8ab2f5196422 None admin] PATH_INFO: `/v3/users/8179b7e759db449e9295e696b2333c9c` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:40.826479 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec85937-1b50-4e30-970d-4c7498bc7a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:40.827082 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec85937-1b50-4e30-970d-4c7498bc7a14 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:40.827355 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec85937-1b50-4e30-970d-4c7498bc7a14 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:40.827473 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591c8f97-a046-4590-a73f-8ab2f5196422 None admin] RBAC: Authorizing `identity:get_user(user_id=8179b7e759db449e9295e696b2333c9c)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:40.827621 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec85937-1b50-4e30-970d-4c7498bc7a14 None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:40.828278 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec85937-1b50-4e30-970d-4c7498bc7a14 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:22:40.828577 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec85937-1b50-4e30-970d-4c7498bc7a14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:40.828942 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591c8f97-a046-4590-a73f-8ab2f5196422 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:40.829970 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec85937-1b50-4e30-970d-4c7498bc7a14 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:40.830853 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 39/78] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Aug 10 08:22:40 2022] GET /identity/v3/users/8179b7e759db449e9295e696b2333c9c => generated 275 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:22:40.831292 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591c8f97-a046-4590-a73f-8ab2f5196422 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:40.841277 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 40/79] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Aug 10 08:22:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:22:40.842019 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec85937-1b50-4e30-970d-4c7498bc7a14 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:40.854513 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:40.936746 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591c8f97-a046-4590-a73f-8ab2f5196422 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:40.941868 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec85937-1b50-4e30-970d-4c7498bc7a14 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:40.943368 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:40.969186 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a9ae35-3a1d-401e-b8bb-b032bfbe0966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:40.969814 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a9ae35-3a1d-401e-b8bb-b032bfbe0966 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:40.970101 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a9ae35-3a1d-401e-b8bb-b032bfbe0966 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:40.970402 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a9ae35-3a1d-401e-b8bb-b032bfbe0966 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:40.975256 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb2c677-0d2e-4906-a63c-72a30e675a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:40.976497 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb2c677-0d2e-4906-a63c-72a30e675a5e None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:40.976783 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb2c677-0d2e-4906-a63c-72a30e675a5e None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:40.977086 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb2c677-0d2e-4906-a63c-72a30e675a5e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:40.981026 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a9ae35-3a1d-401e-b8bb-b032bfbe0966 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 08:22:40.985821 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-e3a9ae35-3a1d-401e-b8bb-b032bfbe0966 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 08:22:40.985821 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:22:40.985821 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:22:40.985821 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:22:40.985821 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:22:40.985821 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:22:40.985821 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:22:40.985821 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:22:40.985821 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:22:40.985821 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:22:40.985821 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:22:40.985821 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:22:40.985821 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 08:22:40.985821 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 08:22:40.985821 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 08:22:40.985821 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 08:22:40.986902 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 08:22:40.986902 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 08:22:40.986902 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 08:22:40.986902 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 08:22:40.986902 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:22:40.986902 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:22:40.986902 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:22:40.986902 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:22:40.986902 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:22:40.986902 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:22:40.986902 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:22:40.986902 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:22:40.986902 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:22:40.986902 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:22:40.986902 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:22:40.986902 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:22:40.987801 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:22:40.987801 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:22:40.987801 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:22:40.987801 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:22:40.987801 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 08:22:40.987801 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 08:22:40.987801 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 08:22:40.987801 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 08:22:40.987801 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 08:22:40.987801 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 08:22:40.987801 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 08:22:40.987801 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:22:40.987801 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 40/80] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Aug 10 08:22:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:22:40.987801 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a9ae35-3a1d-401e-b8bb-b032bfbe0966 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:41.047145 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb2c677-0d2e-4906-a63c-72a30e675a5e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:41.048717 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb2c677-0d2e-4906-a63c-72a30e675a5e None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:41.068779 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:41.090380 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a9ae35-3a1d-401e-b8bb-b032bfbe0966 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:41.124535 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166d88d1-b0c3-418d-a22c-79bd4b52a20a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:41.125152 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166d88d1-b0c3-418d-a22c-79bd4b52a20a None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:41.125414 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166d88d1-b0c3-418d-a22c-79bd4b52a20a None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:41.125685 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166d88d1-b0c3-418d-a22c-79bd4b52a20a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:41.126796 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166d88d1-b0c3-418d-a22c-79bd4b52a20a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:41.128251 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166d88d1-b0c3-418d-a22c-79bd4b52a20a None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:41.129842 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 41/81] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:22:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:22:41.130603 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166d88d1-b0c3-418d-a22c-79bd4b52a20a None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:41.149639 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 41/82] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Aug 10 08:22:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 08:22:41.153874 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb2c677-0d2e-4906-a63c-72a30e675a5e None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:41.156193 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:41.233179 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166d88d1-b0c3-418d-a22c-79bd4b52a20a None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:41.251352 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb2c677-0d2e-4906-a63c-72a30e675a5e None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:41.267014 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83636f57-2d4a-4a4e-8bc9-e2bf13a26095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:41.267607 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83636f57-2d4a-4a4e-8bc9-e2bf13a26095 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:41.267882 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83636f57-2d4a-4a4e-8bc9-e2bf13a26095 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:41.268154 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83636f57-2d4a-4a4e-8bc9-e2bf13a26095 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:41.283540 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:41.284536 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7482a5ce-e52f-4b2c-809e-4588f09b89ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:41.285195 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7482a5ce-e52f-4b2c-809e-4588f09b89ed None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:41.285478 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7482a5ce-e52f-4b2c-809e-4588f09b89ed None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:41.285753 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7482a5ce-e52f-4b2c-809e-4588f09b89ed None admin] PATH_INFO: `/v3/users/a79bffd3ad024a2f938d1f77cfd728d5` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:41.286885 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7482a5ce-e52f-4b2c-809e-4588f09b89ed None admin] RBAC: Authorizing `identity:get_user(user_id=a79bffd3ad024a2f938d1f77cfd728d5)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:41.289119 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7482a5ce-e52f-4b2c-809e-4588f09b89ed None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:41.293894 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 42/83] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Aug 10 08:22:41 2022] GET /identity/v3/users/a79bffd3ad024a2f938d1f77cfd728d5 => generated 247 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:22:41.293894 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7482a5ce-e52f-4b2c-809e-4588f09b89ed None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:41.337430 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83636f57-2d4a-4a4e-8bc9-e2bf13a26095 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:41.338912 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83636f57-2d4a-4a4e-8bc9-e2bf13a26095 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:41.360838 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:41.390944 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7482a5ce-e52f-4b2c-809e-4588f09b89ed None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:41.429427 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4753550a-85ac-467d-992e-295ab025a214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:41.430034 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4753550a-85ac-467d-992e-295ab025a214 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:41.430316 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4753550a-85ac-467d-992e-295ab025a214 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:41.430589 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4753550a-85ac-467d-992e-295ab025a214 None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:41.431251 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4753550a-85ac-467d-992e-295ab025a214 None admin] RBAC: Adding query filter params (name=member) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:22:41.431555 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4753550a-85ac-467d-992e-295ab025a214 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:41.432940 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4753550a-85ac-467d-992e-295ab025a214 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:41.435320 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 42/84] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Aug 10 08:22:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 08:22:41.436128 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83636f57-2d4a-4a4e-8bc9-e2bf13a26095 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:41.448437 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 43/85] 10.222.0.57 () {56 vars in 1101 bytes} [Wed Aug 10 08:22:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:22:41.448437 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4753550a-85ac-467d-992e-295ab025a214 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:41.481654 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:41.536837 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83636f57-2d4a-4a4e-8bc9-e2bf13a26095 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:41.545672 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4753550a-85ac-467d-992e-295ab025a214 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:41.571019 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cbf6be-de97-46f1-a763-d724bf61fe45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:41.571613 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cbf6be-de97-46f1-a763-d724bf61fe45 None admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:41.571892 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cbf6be-de97-46f1-a763-d724bf61fe45 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:41.572159 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cbf6be-de97-46f1-a763-d724bf61fe45 None admin] PATH_INFO: `/v3/users` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:41.572790 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cbf6be-de97-46f1-a763-d724bf61fe45 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:41.574252 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cbf6be-de97-46f1-a763-d724bf61fe45 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:41.575279 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:41.579975 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81af384-6786-4b0f-ba22-5c6bd2f93e51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:41.580601 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81af384-6786-4b0f-ba22-5c6bd2f93e51 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:41.580883 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81af384-6786-4b0f-ba22-5c6bd2f93e51 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:41.581184 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81af384-6786-4b0f-ba22-5c6bd2f93e51 None admin] PATH_INFO: `/v3/projects/0f2b0269b8b24890b55cca8078954ba1` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:41.582320 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81af384-6786-4b0f-ba22-5c6bd2f93e51 None admin] RBAC: Authorizing `identity:get_project(project_id=0f2b0269b8b24890b55cca8078954ba1)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:41.583762 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81af384-6786-4b0f-ba22-5c6bd2f93e51 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:41.585463 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 44/86] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Aug 10 08:22:41 2022] GET /identity/v3/projects/0f2b0269b8b24890b55cca8078954ba1 => generated 298 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:22:41.586224 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81af384-6786-4b0f-ba22-5c6bd2f93e51 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:41.662374 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 43/87] 10.222.0.57 () {60 vars in 1132 bytes} [Wed Aug 10 08:22:41 2022] POST /identity/v3/users => generated 289 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:22:41.663829 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cbf6be-de97-46f1-a763-d724bf61fe45 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:41.689069 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81af384-6786-4b0f-ba22-5c6bd2f93e51 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:41.728928 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad26ca8-3f10-4fa6-b828-13b8112fd3af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:41.729667 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad26ca8-3f10-4fa6-b828-13b8112fd3af None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:41.729945 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad26ca8-3f10-4fa6-b828-13b8112fd3af None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:41.730243 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad26ca8-3f10-4fa6-b828-13b8112fd3af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:41.780515 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cbf6be-de97-46f1-a763-d724bf61fe45 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:41.782274 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:41.809845 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:41.813191 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad26ca8-3f10-4fa6-b828-13b8112fd3af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:41.814698 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad26ca8-3f10-4fa6-b828-13b8112fd3af None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:41.823942 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9711b1eb-c220-4767-99d7-ad41190783cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:41.826672 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9711b1eb-c220-4767-99d7-ad41190783cc None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:41.826672 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9711b1eb-c220-4767-99d7-ad41190783cc None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:41.826672 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9711b1eb-c220-4767-99d7-ad41190783cc None admin] PATH_INFO: `/v3/projects/0f2b0269b8b24890b55cca8078954ba1` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:41.828143 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9711b1eb-c220-4767-99d7-ad41190783cc None admin] RBAC: Authorizing `identity:get_project(project_id=0f2b0269b8b24890b55cca8078954ba1)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:41.830858 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9711b1eb-c220-4767-99d7-ad41190783cc None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:41.833462 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 44/88] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Aug 10 08:22:41 2022] GET /identity/v3/projects/0f2b0269b8b24890b55cca8078954ba1 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:22:41.834419 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9711b1eb-c220-4767-99d7-ad41190783cc None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:41.949097 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 45/89] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Aug 10 08:22:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 08:22:41.949885 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad26ca8-3f10-4fa6-b828-13b8112fd3af None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:41.957702 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9711b1eb-c220-4767-99d7-ad41190783cc None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:42.002030 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a19332-193a-4fb2-b186-11a89ef8a1c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:42.002829 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a19332-193a-4fb2-b186-11a89ef8a1c8 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:42.003121 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a19332-193a-4fb2-b186-11a89ef8a1c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:42.003428 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a19332-193a-4fb2-b186-11a89ef8a1c8 None admin] PATH_INFO: `/v3/users/8179b7e759db449e9295e696b2333c9c` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:42.004612 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a19332-193a-4fb2-b186-11a89ef8a1c8 None admin] RBAC: Authorizing `identity:get_user(user_id=8179b7e759db449e9295e696b2333c9c)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:42.006110 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a19332-193a-4fb2-b186-11a89ef8a1c8 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:42.012203 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 45/90] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Aug 10 08:22:41 2022] GET /identity/v3/users/8179b7e759db449e9295e696b2333c9c => generated 275 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:22:42.012203 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a19332-193a-4fb2-b186-11a89ef8a1c8 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:42.073398 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:42.075969 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:42.078824 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad26ca8-3f10-4fa6-b828-13b8112fd3af None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:42.115552 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab96abe-617e-4a8b-95bf-e9fc058d01ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:42.116820 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab96abe-617e-4a8b-95bf-e9fc058d01ea None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:42.117094 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab96abe-617e-4a8b-95bf-e9fc058d01ea None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:42.117337 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab96abe-617e-4a8b-95bf-e9fc058d01ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:42.118240 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab96abe-617e-4a8b-95bf-e9fc058d01ea None admin] RBAC: Adding query filter params (role.id=44b64db484f24a46bf0b7ced46b519bf, scope.project.id=0f2b0269b8b24890b55cca8078954ba1, user.id=8179b7e759db449e9295e696b2333c9c) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:22:42.118548 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab96abe-617e-4a8b-95bf-e9fc058d01ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:42.120743 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab96abe-617e-4a8b-95bf-e9fc058d01ea None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:42.125365 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a19332-193a-4fb2-b186-11a89ef8a1c8 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:42.131565 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 46/91] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Aug 10 08:22:41 2022] GET /identity/v3/role_assignments?user.id=8179b7e759db449e9295e696b2333c9c&scope.project.id=0f2b0269b8b24890b55cca8078954ba1&role.id=44b64db484f24a46bf0b7ced46b519bf => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:22:42.133450 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab96abe-617e-4a8b-95bf-e9fc058d01ea None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:42.168281 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466315f7-ae36-42b1-b86f-8a598a247044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:42.169397 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466315f7-ae36-42b1-b86f-8a598a247044 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:42.170272 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466315f7-ae36-42b1-b86f-8a598a247044 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:42.170990 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466315f7-ae36-42b1-b86f-8a598a247044 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:42.171887 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466315f7-ae36-42b1-b86f-8a598a247044 None admin] RBAC: Adding query filter params (role.id=ccfc8d3fdf2d458299df29001278930f, scope.project.id=0f2b0269b8b24890b55cca8078954ba1, user.id=8179b7e759db449e9295e696b2333c9c) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:22:42.172573 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466315f7-ae36-42b1-b86f-8a598a247044 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:42.174952 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466315f7-ae36-42b1-b86f-8a598a247044 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:42.184497 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 46/92] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Aug 10 08:22:42 2022] GET /identity/v3/role_assignments?user.id=8179b7e759db449e9295e696b2333c9c&scope.project.id=0f2b0269b8b24890b55cca8078954ba1&role.id=ccfc8d3fdf2d458299df29001278930f => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:22:42.185750 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466315f7-ae36-42b1-b86f-8a598a247044 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:42.279297 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab96abe-617e-4a8b-95bf-e9fc058d01ea None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:42.325457 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25f15f2-5a6d-40a4-990c-3f4e3fe2596f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:42.326317 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25f15f2-5a6d-40a4-990c-3f4e3fe2596f None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:42.327350 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25f15f2-5a6d-40a4-990c-3f4e3fe2596f None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:42.327748 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25f15f2-5a6d-40a4-990c-3f4e3fe2596f None admin] PATH_INFO: `/v3/projects/0f2b0269b8b24890b55cca8078954ba1` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:42.329629 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25f15f2-5a6d-40a4-990c-3f4e3fe2596f None admin] RBAC: Authorizing `identity:get_project(project_id=0f2b0269b8b24890b55cca8078954ba1)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:42.331566 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25f15f2-5a6d-40a4-990c-3f4e3fe2596f None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:42.334256 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 47/93] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Aug 10 08:22:42 2022] GET /identity/v3/projects/0f2b0269b8b24890b55cca8078954ba1 => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:22:42.341026 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25f15f2-5a6d-40a4-990c-3f4e3fe2596f None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:42.415424 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:42.427508 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466315f7-ae36-42b1-b86f-8a598a247044 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:42.435162 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:42.494740 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f436c6-8891-4a32-af0c-9cd1d23da178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:42.495362 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f436c6-8891-4a32-af0c-9cd1d23da178 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:42.495645 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f436c6-8891-4a32-af0c-9cd1d23da178 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:42.495917 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f436c6-8891-4a32-af0c-9cd1d23da178 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:42.587229 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f436c6-8891-4a32-af0c-9cd1d23da178 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:42.588775 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f436c6-8891-4a32-af0c-9cd1d23da178 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:42.595083 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25f15f2-5a6d-40a4-990c-3f4e3fe2596f None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:42.685653 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4627c609-a4eb-480f-b1c7-8f0ab68e33fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:42.686290 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4627c609-a4eb-480f-b1c7-8f0ab68e33fd None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:42.686575 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4627c609-a4eb-480f-b1c7-8f0ab68e33fd None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:42.686850 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4627c609-a4eb-480f-b1c7-8f0ab68e33fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:42.687599 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4627c609-a4eb-480f-b1c7-8f0ab68e33fd None admin] RBAC: Adding query filter params (role.id=ec3751692d9c4ecf81b4a71c936c46be, scope.project.id=0f2b0269b8b24890b55cca8078954ba1, user.id=a79bffd3ad024a2f938d1f77cfd728d5) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:22:42.687911 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4627c609-a4eb-480f-b1c7-8f0ab68e33fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:42.689425 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4627c609-a4eb-480f-b1c7-8f0ab68e33fd None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:42.699909 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 48/94] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Aug 10 08:22:42 2022] GET /identity/v3/role_assignments?user.id=a79bffd3ad024a2f938d1f77cfd728d5&scope.project.id=0f2b0269b8b24890b55cca8078954ba1&role.id=ec3751692d9c4ecf81b4a71c936c46be => generated 259 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:22:42.702770 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 47/95] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Aug 10 08:22:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 517 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 08:22:42.708993 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4627c609-a4eb-480f-b1c7-8f0ab68e33fd None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:42.844588 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:42.852293 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4627c609-a4eb-480f-b1c7-8f0ab68e33fd None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:42.900764 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189737b5-2d33-445b-99da-36429e2a3eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:42.901445 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189737b5-2d33-445b-99da-36429e2a3eaf None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:42.901785 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189737b5-2d33-445b-99da-36429e2a3eaf None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:42.902071 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189737b5-2d33-445b-99da-36429e2a3eaf None admin] PATH_INFO: `/v3/projects/fb99099b189242a3bc6159b227bc4e21` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:42.905626 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189737b5-2d33-445b-99da-36429e2a3eaf None admin] RBAC: Authorizing `identity:get_project(project_id=fb99099b189242a3bc6159b227bc4e21)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:42.907106 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189737b5-2d33-445b-99da-36429e2a3eaf None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:42.909199 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 49/96] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Aug 10 08:22:42 2022] GET /identity/v3/projects/fb99099b189242a3bc6159b227bc4e21 => generated 312 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:22:42.915924 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f436c6-8891-4a32-af0c-9cd1d23da178 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:42.918078 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:43.041695 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f436c6-8891-4a32-af0c-9cd1d23da178 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:43.082334 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b314f29d-943e-4c91-b8d7-ce68a5ddfebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:43.082950 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b314f29d-943e-4c91-b8d7-ce68a5ddfebf None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:43.083231 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b314f29d-943e-4c91-b8d7-ce68a5ddfebf None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:43.083563 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b314f29d-943e-4c91-b8d7-ce68a5ddfebf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:43.084294 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b314f29d-943e-4c91-b8d7-ce68a5ddfebf None admin] RBAC: Adding query filter params (role.id=44b64db484f24a46bf0b7ced46b519bf, scope.project.id=fb99099b189242a3bc6159b227bc4e21, user.id=8179b7e759db449e9295e696b2333c9c) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:22:43.084604 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b314f29d-943e-4c91-b8d7-ce68a5ddfebf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:43.086944 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b314f29d-943e-4c91-b8d7-ce68a5ddfebf None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:43.093028 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 48/97] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Aug 10 08:22:42 2022] GET /identity/v3/role_assignments?user.id=8179b7e759db449e9295e696b2333c9c&scope.project.id=fb99099b189242a3bc6159b227bc4e21&role.id=44b64db484f24a46bf0b7ced46b519bf => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:22:44.838103 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0c88e8-a93c-425c-a787-e82b936d1c6b None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:44.838625 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0c88e8-a93c-425c-a787-e82b936d1c6b None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:44.838625 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0c88e8-a93c-425c-a787-e82b936d1c6b None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:44.840284 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 50/98] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:22:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:22:44.845495 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b881d05c-8871-41c8-8a04-3b6361b72b40 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:44.845912 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b881d05c-8871-41c8-8a04-3b6361b72b40 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:44.846101 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b881d05c-8871-41c8-8a04-3b6361b72b40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:44.865410 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-b881d05c-8871-41c8-8a04-3b6361b72b40 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:22:44.866821 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-b881d05c-8871-41c8-8a04-3b6361b72b40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:22:44.909065 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457e63f5-66e7-4326-8a9b-cca381156038 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:44.909386 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457e63f5-66e7-4326-8a9b-cca381156038 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:44.909619 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457e63f5-66e7-4326-8a9b-cca381156038 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:44.911185 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 51/99] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:22:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 08:22:44.922016 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc943b8d-222e-46ff-875d-711e46c86f86 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:44.922295 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc943b8d-222e-46ff-875d-711e46c86f86 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:44.922569 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc943b8d-222e-46ff-875d-711e46c86f86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:44.945168 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-dc943b8d-222e-46ff-875d-711e46c86f86 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:22:44.946419 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:44.948072 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-dc943b8d-222e-46ff-875d-711e46c86f86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:22:44.956793 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 49/100] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:22:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 08:22:45.039340 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b881d05c-8871-41c8-8a04-3b6361b72b40 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:45.043182 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 52/101] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:22:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 08:22:45.079425 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc943b8d-222e-46ff-875d-711e46c86f86 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:45.204781 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b881d05c-8871-41c8-8a04-3b6361b72b40 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:45.209147 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:45.239030 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc943b8d-222e-46ff-875d-711e46c86f86 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:45.249533 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef357f15-6fe5-4f69-b385-89d8ef901fe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:45.250719 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef357f15-6fe5-4f69-b385-89d8ef901fe1 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:45.250999 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef357f15-6fe5-4f69-b385-89d8ef901fe1 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:45.251273 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef357f15-6fe5-4f69-b385-89d8ef901fe1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:45.252423 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef357f15-6fe5-4f69-b385-89d8ef901fe1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:45.253947 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef357f15-6fe5-4f69-b385-89d8ef901fe1 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:45.255739 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 50/102] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:22:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:22:45.256691 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16306523-4d1a-4198-bbd1-23c63dc03f45 None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:45.256976 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16306523-4d1a-4198-bbd1-23c63dc03f45 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:45.257260 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16306523-4d1a-4198-bbd1-23c63dc03f45 None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:45.258619 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 51/103] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:22:45 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:22:45.263449 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16306523-4d1a-4198-bbd1-23c63dc03f45 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:45.304318 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3fd10d-9e28-44d6-9e3b-1f46d1bf931e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:45.305145 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3fd10d-9e28-44d6-9e3b-1f46d1bf931e None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:45.305432 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3fd10d-9e28-44d6-9e3b-1f46d1bf931e None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:45.305704 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3fd10d-9e28-44d6-9e3b-1f46d1bf931e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:45.316333 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3fd10d-9e28-44d6-9e3b-1f46d1bf931e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 08:22:45.321242 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-3f3fd10d-9e28-44d6-9e3b-1f46d1bf931e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 08:22:45.321242 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:22:45.321242 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:22:45.321242 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:22:45.321242 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:22:45.321242 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:22:45.321242 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:22:45.321242 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:22:45.321242 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:22:45.321242 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:22:45.321242 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:22:45.321242 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:22:45.321242 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 08:22:45.321242 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 08:22:45.321242 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 08:22:45.321242 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 08:22:45.322124 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 08:22:45.322124 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 08:22:45.322124 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 08:22:45.322124 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 08:22:45.322124 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:22:45.322124 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:22:45.322124 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:22:45.322124 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:22:45.322124 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:22:45.322124 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:22:45.322124 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:22:45.322124 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:22:45.322124 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:22:45.322124 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:22:45.322124 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:22:45.322124 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:22:45.322911 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:22:45.322911 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:22:45.322911 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:22:45.322911 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:22:45.322911 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 08:22:45.322911 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 08:22:45.322911 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 08:22:45.322911 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 08:22:45.322911 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 08:22:45.322911 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 08:22:45.322911 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 08:22:45.322911 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:22:45.322911 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 53/104] 10.222.0.57 () {56 vars in 1114 bytes} [Wed Aug 10 08:22:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:22:45.323568 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b584817-0bc4-49ac-b9f8-2589433d7f9c None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:45.324052 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b584817-0bc4-49ac-b9f8-2589433d7f9c None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:45.324052 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b584817-0bc4-49ac-b9f8-2589433d7f9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:45.339732 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:45.347207 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-0b584817-0bc4-49ac-b9f8-2589433d7f9c None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:22:45.348512 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-0b584817-0bc4-49ac-b9f8-2589433d7f9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:22:45.392086 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16306523-4d1a-4198-bbd1-23c63dc03f45 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:45.430866 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 54/105] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:22:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 08:22:45.432487 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5147d2-65b6-496d-afca-5b111a7b8d3e None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:45.433053 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5147d2-65b6-496d-afca-5b111a7b8d3e None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:45.433483 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5147d2-65b6-496d-afca-5b111a7b8d3e None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:45.435618 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 55/106] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:22:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:22:45.436655 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5147d2-65b6-496d-afca-5b111a7b8d3e None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:45.445752 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df93c13c-109c-4b95-84ec-cb59cd41ae19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:45.446551 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df93c13c-109c-4b95-84ec-cb59cd41ae19 None admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:45.446907 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df93c13c-109c-4b95-84ec-cb59cd41ae19 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:45.447259 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df93c13c-109c-4b95-84ec-cb59cd41ae19 None admin] PATH_INFO: `/v3/projects` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:45.448049 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df93c13c-109c-4b95-84ec-cb59cd41ae19 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:45.449771 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df93c13c-109c-4b95-84ec-cb59cd41ae19 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:45.527118 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:45.536533 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 52/107] 10.222.0.57 () {60 vars in 1141 bytes} [Wed Aug 10 08:22:45 2022] POST /identity/v3/projects => generated 302 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:22:45.545199 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5147d2-65b6-496d-afca-5b111a7b8d3e None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:45.546465 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2ec86c-0350-486d-b768-de09a89f6474 None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:45.546763 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2ec86c-0350-486d-b768-de09a89f6474 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:45.547071 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2ec86c-0350-486d-b768-de09a89f6474 None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:45.548424 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 53/108] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:22:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:22:45.555163 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cb4cf2-0a49-47ab-9687-45d5008d51d0 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:45.555452 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cb4cf2-0a49-47ab-9687-45d5008d51d0 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:45.555734 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cb4cf2-0a49-47ab-9687-45d5008d51d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:45.578888 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-56cb4cf2-0a49-47ab-9687-45d5008d51d0 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:22:45.580982 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-56cb4cf2-0a49-47ab-9687-45d5008d51d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:22:45.595072 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7a6a06-23b9-4022-84fc-d7a096714908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:45.595880 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7a6a06-23b9-4022-84fc-d7a096714908 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:45.596248 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7a6a06-23b9-4022-84fc-d7a096714908 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:45.596688 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7a6a06-23b9-4022-84fc-d7a096714908 None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:45.597673 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7a6a06-23b9-4022-84fc-d7a096714908 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:22:45.598206 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7a6a06-23b9-4022-84fc-d7a096714908 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:45.599843 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7a6a06-23b9-4022-84fc-d7a096714908 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:45.611857 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 56/109] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Aug 10 08:22:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:22:45.617340 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7a6a06-23b9-4022-84fc-d7a096714908 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:45.673616 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 54/110] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:22:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 08:22:45.680097 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4a435b-114f-4a38-a7d8-6cdc008db344 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:45.680515 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4a435b-114f-4a38-a7d8-6cdc008db344 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:45.680972 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4a435b-114f-4a38-a7d8-6cdc008db344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:45.718259 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-0c4a435b-114f-4a38-a7d8-6cdc008db344 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:22:45.719454 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-0c4a435b-114f-4a38-a7d8-6cdc008db344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:22:45.746990 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7a6a06-23b9-4022-84fc-d7a096714908 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:45.779550 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:45.797239 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52468a88-0e61-4b15-9e05-700a5bc86027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:45.800258 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52468a88-0e61-4b15-9e05-700a5bc86027 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:45.800258 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52468a88-0e61-4b15-9e05-700a5bc86027 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:45.800258 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52468a88-0e61-4b15-9e05-700a5bc86027 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:45.816942 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 55/111] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:22:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 08:22:45.816942 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4a435b-114f-4a38-a7d8-6cdc008db344 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:45.824860 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52468a88-0e61-4b15-9e05-700a5bc86027 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 08:22:45.835827 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-52468a88-0e61-4b15-9e05-700a5bc86027 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 08:22:45.835827 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:22:45.835827 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:22:45.835827 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:22:45.835827 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:22:45.835827 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:22:45.835827 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:22:45.835827 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:22:45.835827 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:22:45.835827 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:22:45.835827 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:22:45.835827 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:22:45.835827 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 08:22:45.835827 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 08:22:45.835827 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 08:22:45.835827 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 08:22:45.836723 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 08:22:45.836723 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 08:22:45.836723 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 08:22:45.836723 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 08:22:45.836723 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:22:45.836723 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:22:45.836723 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:22:45.836723 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:22:45.836723 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:22:45.836723 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:22:45.836723 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:22:45.836723 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:22:45.836723 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:22:45.836723 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:22:45.836723 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:22:45.836723 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:22:45.837650 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:22:45.837650 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:22:45.837650 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:22:45.837650 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:22:45.837650 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 08:22:45.837650 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 08:22:45.837650 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 08:22:45.837650 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 08:22:45.837650 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 08:22:45.837650 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 08:22:45.837650 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 08:22:45.837650 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:22:45.837650 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 57/112] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Aug 10 08:22:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:22:45.838758 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52468a88-0e61-4b15-9e05-700a5bc86027 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:45.853943 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:45.959188 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4a435b-114f-4a38-a7d8-6cdc008db344 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:45.981978 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52468a88-0e61-4b15-9e05-700a5bc86027 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:46.000641 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97598b93-ddcb-4535-b4a0-d43d6abf6df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:46.001599 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97598b93-ddcb-4535-b4a0-d43d6abf6df2 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:46.002080 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97598b93-ddcb-4535-b4a0-d43d6abf6df2 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:46.002576 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97598b93-ddcb-4535-b4a0-d43d6abf6df2 None admin] PATH_INFO: `/v3/users/8179b7e759db449e9295e696b2333c9c` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:46.003943 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97598b93-ddcb-4535-b4a0-d43d6abf6df2 None admin] RBAC: Authorizing `identity:get_user(user_id=8179b7e759db449e9295e696b2333c9c)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:46.005659 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97598b93-ddcb-4535-b4a0-d43d6abf6df2 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:46.008215 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 56/113] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Aug 10 08:22:45 2022] GET /identity/v3/users/8179b7e759db449e9295e696b2333c9c => generated 275 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:22:46.009235 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97598b93-ddcb-4535-b4a0-d43d6abf6df2 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:46.027132 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8359c6dd-14fe-4fcc-ab0e-e1e8c708cd9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:46.028057 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8359c6dd-14fe-4fcc-ab0e-e1e8c708cd9c None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:46.028343 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8359c6dd-14fe-4fcc-ab0e-e1e8c708cd9c None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:46.028618 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8359c6dd-14fe-4fcc-ab0e-e1e8c708cd9c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:46.043225 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8359c6dd-14fe-4fcc-ab0e-e1e8c708cd9c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 08:22:46.050082 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-8359c6dd-14fe-4fcc-ab0e-e1e8c708cd9c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 08:22:46.050082 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:22:46.050082 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:22:46.050082 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:22:46.050082 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:22:46.050082 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:22:46.050082 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:22:46.050082 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:22:46.050082 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:22:46.050082 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:22:46.050082 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:22:46.050082 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:22:46.050082 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 08:22:46.050082 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 08:22:46.050082 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 08:22:46.050082 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 08:22:46.050947 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 08:22:46.050947 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 08:22:46.050947 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 08:22:46.050947 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 08:22:46.050947 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:22:46.050947 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:22:46.050947 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:22:46.050947 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:22:46.050947 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:22:46.050947 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:22:46.050947 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:22:46.050947 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:22:46.050947 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:22:46.050947 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:22:46.050947 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:22:46.050947 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:22:46.051849 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:22:46.051849 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:22:46.051849 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:22:46.051849 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:22:46.051849 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 08:22:46.051849 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 08:22:46.051849 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 08:22:46.051849 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 08:22:46.051849 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 08:22:46.051849 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 08:22:46.051849 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 08:22:46.051849 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:22:46.051849 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 58/114] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Aug 10 08:22:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:22:46.052509 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8359c6dd-14fe-4fcc-ab0e-e1e8c708cd9c None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:46.068491 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:46.139862 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97598b93-ddcb-4535-b4a0-d43d6abf6df2 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:46.141830 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:46.177672 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8359c6dd-14fe-4fcc-ab0e-e1e8c708cd9c None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:46.182012 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fd128d-6a37-47e9-8b74-21fec1d9a46e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:46.182684 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fd128d-6a37-47e9-8b74-21fec1d9a46e None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:46.182968 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fd128d-6a37-47e9-8b74-21fec1d9a46e None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:46.183239 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fd128d-6a37-47e9-8b74-21fec1d9a46e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:46.193970 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fd128d-6a37-47e9-8b74-21fec1d9a46e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 08:22:46.199632 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-e7fd128d-6a37-47e9-8b74-21fec1d9a46e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 08:22:46.199632 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:22:46.199632 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:22:46.199632 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:22:46.199632 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:22:46.199632 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:22:46.199632 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:22:46.199632 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:22:46.199632 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:22:46.199632 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:22:46.199632 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:22:46.199632 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:22:46.199632 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 08:22:46.199632 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 08:22:46.199632 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 08:22:46.199632 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 08:22:46.200515 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 08:22:46.200515 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 08:22:46.200515 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 08:22:46.200515 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 08:22:46.200515 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:22:46.200515 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:22:46.200515 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:22:46.200515 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:22:46.200515 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:22:46.200515 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:22:46.200515 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:22:46.200515 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:22:46.200515 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:22:46.200515 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:22:46.200515 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:22:46.200515 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:22:46.201320 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:22:46.201320 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:22:46.201320 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:22:46.201320 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:22:46.201320 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 08:22:46.201320 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 08:22:46.201320 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 08:22:46.201320 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 08:22:46.201320 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 08:22:46.201320 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 08:22:46.201320 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 08:22:46.201320 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:22:46.201320 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 57/115] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Aug 10 08:22:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:22:46.205618 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fd128d-6a37-47e9-8b74-21fec1d9a46e None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:46.219952 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f7bdd5-a9b3-4568-be83-4c232d062324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:46.220561 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f7bdd5-a9b3-4568-be83-4c232d062324 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:46.220842 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f7bdd5-a9b3-4568-be83-4c232d062324 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:46.221250 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f7bdd5-a9b3-4568-be83-4c232d062324 None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:46.222025 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f7bdd5-a9b3-4568-be83-4c232d062324 None admin] RBAC: Adding query filter params (name=member) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:22:46.222334 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f7bdd5-a9b3-4568-be83-4c232d062324 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:46.223715 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f7bdd5-a9b3-4568-be83-4c232d062324 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:46.237689 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 59/116] 10.222.0.57 () {56 vars in 1101 bytes} [Wed Aug 10 08:22:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:22:46.237689 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f7bdd5-a9b3-4568-be83-4c232d062324 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:46.311922 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fd128d-6a37-47e9-8b74-21fec1d9a46e None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:46.314272 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:46.345813 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f7bdd5-a9b3-4568-be83-4c232d062324 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:46.348163 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b67455-d3f7-49a7-a1cc-38886dcb4d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:46.348806 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b67455-d3f7-49a7-a1cc-38886dcb4d0d None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:46.349190 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b67455-d3f7-49a7-a1cc-38886dcb4d0d None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:46.349470 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b67455-d3f7-49a7-a1cc-38886dcb4d0d None admin] PATH_INFO: `/v3/projects/0f2b0269b8b24890b55cca8078954ba1` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:46.350672 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b67455-d3f7-49a7-a1cc-38886dcb4d0d None admin] RBAC: Authorizing `identity:get_project(project_id=0f2b0269b8b24890b55cca8078954ba1)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:46.352151 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b67455-d3f7-49a7-a1cc-38886dcb4d0d None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:46.358478 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 58/117] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Aug 10 08:22:46 2022] GET /identity/v3/projects/0f2b0269b8b24890b55cca8078954ba1 => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:22:46.358478 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad5b3be-ad69-4c55-b369-fa60d5fcd3b1 None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:46.358478 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad5b3be-ad69-4c55-b369-fa60d5fcd3b1 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:46.358478 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad5b3be-ad69-4c55-b369-fa60d5fcd3b1 None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:46.358478 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 59/118] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:22:46 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:22:46.358478 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad5b3be-ad69-4c55-b369-fa60d5fcd3b1 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:46.380770 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:46.392611 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557652a2-6598-4515-a202-f70d9b6db629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:46.393471 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557652a2-6598-4515-a202-f70d9b6db629 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:46.393905 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557652a2-6598-4515-a202-f70d9b6db629 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:46.394180 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557652a2-6598-4515-a202-f70d9b6db629 None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:46.394854 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557652a2-6598-4515-a202-f70d9b6db629 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:22:46.395192 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557652a2-6598-4515-a202-f70d9b6db629 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:46.396623 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557652a2-6598-4515-a202-f70d9b6db629 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:46.406011 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 60/119] 10.222.0.57 () {56 vars in 1101 bytes} [Wed Aug 10 08:22:46 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:22:46.407048 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557652a2-6598-4515-a202-f70d9b6db629 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:46.470380 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad5b3be-ad69-4c55-b369-fa60d5fcd3b1 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:46.507045 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066f20a2-bea5-4799-af27-31bce0087989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:46.507653 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066f20a2-bea5-4799-af27-31bce0087989 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:46.507935 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066f20a2-bea5-4799-af27-31bce0087989 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:46.508207 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066f20a2-bea5-4799-af27-31bce0087989 None admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:46.509110 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066f20a2-bea5-4799-af27-31bce0087989 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:22:46.509417 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066f20a2-bea5-4799-af27-31bce0087989 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:46.510813 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066f20a2-bea5-4799-af27-31bce0087989 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:46.524543 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 60/120] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Aug 10 08:22:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:22:46.524685 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557652a2-6598-4515-a202-f70d9b6db629 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:46.524801 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066f20a2-bea5-4799-af27-31bce0087989 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:46.557976 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:46.560502 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c31b9b-dfe2-40f1-a5bd-00a43aa063a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:46.561161 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c31b9b-dfe2-40f1-a5bd-00a43aa063a3 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:46.561446 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c31b9b-dfe2-40f1-a5bd-00a43aa063a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:46.561721 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c31b9b-dfe2-40f1-a5bd-00a43aa063a3 None admin] PATH_INFO: `/v3/users/8179b7e759db449e9295e696b2333c9c` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:46.562908 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c31b9b-dfe2-40f1-a5bd-00a43aa063a3 None admin] RBAC: Authorizing `identity:get_user(user_id=8179b7e759db449e9295e696b2333c9c)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:46.564315 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c31b9b-dfe2-40f1-a5bd-00a43aa063a3 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:46.570453 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 61/121] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Aug 10 08:22:46 2022] GET /identity/v3/users/8179b7e759db449e9295e696b2333c9c => generated 275 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:22:46.570453 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612ac36f-4794-4d69-a294-ac3dad40f860 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:46.570453 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612ac36f-4794-4d69-a294-ac3dad40f860 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:46.570453 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612ac36f-4794-4d69-a294-ac3dad40f860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:46.586175 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-612ac36f-4794-4d69-a294-ac3dad40f860 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:22:46.587351 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-612ac36f-4794-4d69-a294-ac3dad40f860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:22:46.631697 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066f20a2-bea5-4799-af27-31bce0087989 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:46.646876 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:46.661547 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 62/122] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:22:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 08:22:46.662384 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612ac36f-4794-4d69-a294-ac3dad40f860 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:46.676528 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1201c24f-4fab-461d-806c-e21315307514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:46.677957 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1201c24f-4fab-461d-806c-e21315307514 None admin] REQUEST_METHOD: `PUT` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:46.678869 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1201c24f-4fab-461d-806c-e21315307514 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:46.679824 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1201c24f-4fab-461d-806c-e21315307514 None admin] PATH_INFO: `/v3/projects/0f2b0269b8b24890b55cca8078954ba1/users/8179b7e759db449e9295e696b2333c9c/roles/ccfc8d3fdf2d458299df29001278930f` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:46.683120 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1201c24f-4fab-461d-806c-e21315307514 None admin] RBAC: Authorizing `identity:create_grant(project_id=0f2b0269b8b24890b55cca8078954ba1, user_id=8179b7e759db449e9295e696b2333c9c, role_id=ccfc8d3fdf2d458299df29001278930f)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:46.685721 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1201c24f-4fab-461d-806c-e21315307514 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:46.763670 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 61/123] 10.222.0.57 () {58 vars in 1439 bytes} [Wed Aug 10 08:22:46 2022] PUT /identity/v3/projects/0f2b0269b8b24890b55cca8078954ba1/users/8179b7e759db449e9295e696b2333c9c/roles/ccfc8d3fdf2d458299df29001278930f => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:22:46.764965 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1201c24f-4fab-461d-806c-e21315307514 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:46.776177 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612ac36f-4794-4d69-a294-ac3dad40f860 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:46.817783 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c218843-909a-4a8c-99a6-0204eb3f88fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:46.818407 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c218843-909a-4a8c-99a6-0204eb3f88fe None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:46.818735 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c218843-909a-4a8c-99a6-0204eb3f88fe None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:46.819004 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c218843-909a-4a8c-99a6-0204eb3f88fe None admin] PATH_INFO: `/v3/users/3a70f390119d4c168dc285643fd110cc` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:46.850050 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c218843-909a-4a8c-99a6-0204eb3f88fe None admin] RBAC: Authorizing `identity:get_user(user_id=3a70f390119d4c168dc285643fd110cc)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:46.851844 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c218843-909a-4a8c-99a6-0204eb3f88fe None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:46.857039 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 63/124] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Aug 10 08:22:46 2022] GET /identity/v3/users/3a70f390119d4c168dc285643fd110cc => generated 289 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:22:46.857039 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c218843-909a-4a8c-99a6-0204eb3f88fe None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:46.896136 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:46.903761 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1201c24f-4fab-461d-806c-e21315307514 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:46.923961 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:46.947709 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e79697-def2-4916-bf8a-9fdf2c75d35a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:46.948642 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e79697-def2-4916-bf8a-9fdf2c75d35a None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:46.948940 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e79697-def2-4916-bf8a-9fdf2c75d35a None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:46.949230 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e79697-def2-4916-bf8a-9fdf2c75d35a None admin] PATH_INFO: `/v3/users/a79bffd3ad024a2f938d1f77cfd728d5` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:46.950395 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e79697-def2-4916-bf8a-9fdf2c75d35a None admin] RBAC: Authorizing `identity:get_user(user_id=a79bffd3ad024a2f938d1f77cfd728d5)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:46.952064 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e79697-def2-4916-bf8a-9fdf2c75d35a None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:46.954803 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 62/125] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Aug 10 08:22:46 2022] GET /identity/v3/users/a79bffd3ad024a2f938d1f77cfd728d5 => generated 247 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:22:46.955753 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e79697-def2-4916-bf8a-9fdf2c75d35a None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:47.000145 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c218843-909a-4a8c-99a6-0204eb3f88fe None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:47.041107 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c961b0d-8dea-4391-9725-1cc0f8117a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:47.041750 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c961b0d-8dea-4391-9725-1cc0f8117a7d None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:47.042031 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c961b0d-8dea-4391-9725-1cc0f8117a7d None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:47.042303 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c961b0d-8dea-4391-9725-1cc0f8117a7d None admin] PATH_INFO: `/v3/projects/0f2b0269b8b24890b55cca8078954ba1` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:47.043575 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c961b0d-8dea-4391-9725-1cc0f8117a7d None admin] RBAC: Authorizing `identity:get_project(project_id=0f2b0269b8b24890b55cca8078954ba1)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:47.045085 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c961b0d-8dea-4391-9725-1cc0f8117a7d None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:47.046942 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 64/126] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Aug 10 08:22:46 2022] GET /identity/v3/projects/0f2b0269b8b24890b55cca8078954ba1 => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:22:47.047552 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c961b0d-8dea-4391-9725-1cc0f8117a7d None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:47.085836 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e79697-def2-4916-bf8a-9fdf2c75d35a None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:47.123679 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4655b50-b1e3-4a71-b7cd-cd5223ed2d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:47.124617 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4655b50-b1e3-4a71-b7cd-cd5223ed2d6a None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:47.125024 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4655b50-b1e3-4a71-b7cd-cd5223ed2d6a None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:47.125394 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4655b50-b1e3-4a71-b7cd-cd5223ed2d6a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:47.137732 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4655b50-b1e3-4a71-b7cd-cd5223ed2d6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 08:22:47.143542 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-b4655b50-b1e3-4a71-b7cd-cd5223ed2d6a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 08:22:47.143542 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:22:47.143542 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:22:47.143542 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:22:47.143542 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:22:47.143542 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:22:47.143542 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:22:47.143542 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:22:47.143542 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:22:47.143542 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:22:47.143542 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:22:47.143542 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:22:47.143542 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 08:22:47.143542 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 08:22:47.143542 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 08:22:47.143542 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 08:22:47.144687 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 08:22:47.144687 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 08:22:47.144687 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 08:22:47.144687 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 08:22:47.144687 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:22:47.144687 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:22:47.144687 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:22:47.144687 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:22:47.144687 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:22:47.144687 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:22:47.144687 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:22:47.144687 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:22:47.144687 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:22:47.144687 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:22:47.144687 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:22:47.144687 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:22:47.145718 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:22:47.145718 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:22:47.145718 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:22:47.145718 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:22:47.145718 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 08:22:47.145718 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 08:22:47.145718 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 08:22:47.145718 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 08:22:47.145718 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 08:22:47.145718 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 08:22:47.145718 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 08:22:47.145718 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:22:47.152011 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 63/127] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Aug 10 08:22:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:22:47.152648 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c961b0d-8dea-4391-9725-1cc0f8117a7d None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:47.153141 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4655b50-b1e3-4a71-b7cd-cd5223ed2d6a None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:47.167803 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:47.181368 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:47.189831 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020d153c-b9c9-454c-8a37-4477981106ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:47.190709 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020d153c-b9c9-454c-8a37-4477981106ad None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:47.191090 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020d153c-b9c9-454c-8a37-4477981106ad None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:47.191469 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020d153c-b9c9-454c-8a37-4477981106ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:47.273197 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4655b50-b1e3-4a71-b7cd-cd5223ed2d6a None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:47.290728 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020d153c-b9c9-454c-8a37-4477981106ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:47.292207 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020d153c-b9c9-454c-8a37-4477981106ad None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:47.318486 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c735193-5f66-42b3-9305-143bb0b39571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:47.319758 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c735193-5f66-42b3-9305-143bb0b39571 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:47.320242 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c735193-5f66-42b3-9305-143bb0b39571 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:47.321124 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c735193-5f66-42b3-9305-143bb0b39571 None admin] PATH_INFO: `/v3/projects/0f2b0269b8b24890b55cca8078954ba1` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:47.323046 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c735193-5f66-42b3-9305-143bb0b39571 None admin] RBAC: Authorizing `identity:get_project(project_id=0f2b0269b8b24890b55cca8078954ba1)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:47.325423 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c735193-5f66-42b3-9305-143bb0b39571 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:47.328460 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 64/128] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Aug 10 08:22:47 2022] GET /identity/v3/projects/0f2b0269b8b24890b55cca8078954ba1 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:22:47.329446 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c735193-5f66-42b3-9305-143bb0b39571 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:47.422297 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 65/129] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Aug 10 08:22:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 08:22:47.423231 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020d153c-b9c9-454c-8a37-4477981106ad None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:47.439776 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:47.464677 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:47.478716 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c735193-5f66-42b3-9305-143bb0b39571 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:47.529720 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a9f226-47ec-4535-a710-a19c90ac8ebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:47.531003 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a9f226-47ec-4535-a710-a19c90ac8ebc None admin] REQUEST_METHOD: `PUT` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:47.531707 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a9f226-47ec-4535-a710-a19c90ac8ebc None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:47.532580 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a9f226-47ec-4535-a710-a19c90ac8ebc None admin] PATH_INFO: `/v3/projects/0f2b0269b8b24890b55cca8078954ba1/users/8179b7e759db449e9295e696b2333c9c/roles/44b64db484f24a46bf0b7ced46b519bf` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:47.535066 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020d153c-b9c9-454c-8a37-4477981106ad None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:47.535480 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a9f226-47ec-4535-a710-a19c90ac8ebc None admin] RBAC: Authorizing `identity:create_grant(project_id=0f2b0269b8b24890b55cca8078954ba1, user_id=8179b7e759db449e9295e696b2333c9c, role_id=44b64db484f24a46bf0b7ced46b519bf)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:47.538911 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a9f226-47ec-4535-a710-a19c90ac8ebc None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:47.573309 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442c20c6-a432-4ce8-9ed9-8f8d98b15bed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:47.573919 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442c20c6-a432-4ce8-9ed9-8f8d98b15bed None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:47.574196 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442c20c6-a432-4ce8-9ed9-8f8d98b15bed None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:47.574463 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442c20c6-a432-4ce8-9ed9-8f8d98b15bed None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:47.575118 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442c20c6-a432-4ce8-9ed9-8f8d98b15bed None admin] RBAC: Adding query filter params (name=member) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:22:47.575419 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442c20c6-a432-4ce8-9ed9-8f8d98b15bed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:47.576823 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442c20c6-a432-4ce8-9ed9-8f8d98b15bed None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:47.594054 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 66/130] 10.222.0.57 () {56 vars in 1101 bytes} [Wed Aug 10 08:22:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:22:47.594054 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442c20c6-a432-4ce8-9ed9-8f8d98b15bed None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:47.630657 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 65/131] 10.222.0.57 () {58 vars in 1439 bytes} [Wed Aug 10 08:22:47 2022] PUT /identity/v3/projects/0f2b0269b8b24890b55cca8078954ba1/users/8179b7e759db449e9295e696b2333c9c/roles/44b64db484f24a46bf0b7ced46b519bf => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:22:47.631957 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a9f226-47ec-4535-a710-a19c90ac8ebc None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:47.706260 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:47.734024 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442c20c6-a432-4ce8-9ed9-8f8d98b15bed None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:47.774384 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4210cc-13ae-4ad4-8531-23a256f806d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:47.775434 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4210cc-13ae-4ad4-8531-23a256f806d8 None admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:47.775892 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4210cc-13ae-4ad4-8531-23a256f806d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:47.776338 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4210cc-13ae-4ad4-8531-23a256f806d8 None admin] PATH_INFO: `/v3/projects/0f2b0269b8b24890b55cca8078954ba1/users/a79bffd3ad024a2f938d1f77cfd728d5/roles/ec3751692d9c4ecf81b4a71c936c46be` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:47.778678 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4210cc-13ae-4ad4-8531-23a256f806d8 None admin] RBAC: Authorizing `identity:create_grant(project_id=0f2b0269b8b24890b55cca8078954ba1, user_id=a79bffd3ad024a2f938d1f77cfd728d5, role_id=ec3751692d9c4ecf81b4a71c936c46be)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:47.780272 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4210cc-13ae-4ad4-8531-23a256f806d8 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:47.789316 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a9f226-47ec-4535-a710-a19c90ac8ebc None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:47.830900 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93819a44-a152-4c3e-af04-c02b66a15e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:47.831571 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93819a44-a152-4c3e-af04-c02b66a15e19 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:47.831857 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93819a44-a152-4c3e-af04-c02b66a15e19 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:47.832187 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93819a44-a152-4c3e-af04-c02b66a15e19 None admin] PATH_INFO: `/v3/projects/0f2b0269b8b24890b55cca8078954ba1` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:47.833385 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93819a44-a152-4c3e-af04-c02b66a15e19 None admin] RBAC: Authorizing `identity:get_project(project_id=0f2b0269b8b24890b55cca8078954ba1)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:47.835466 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93819a44-a152-4c3e-af04-c02b66a15e19 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:47.837196 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 66/132] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Aug 10 08:22:47 2022] GET /identity/v3/projects/0f2b0269b8b24890b55cca8078954ba1 => generated 298 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:22:47.838051 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93819a44-a152-4c3e-af04-c02b66a15e19 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:47.857795 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:47.878162 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 67/133] 10.222.0.57 () {58 vars in 1439 bytes} [Wed Aug 10 08:22:47 2022] PUT /identity/v3/projects/0f2b0269b8b24890b55cca8078954ba1/users/a79bffd3ad024a2f938d1f77cfd728d5/roles/ec3751692d9c4ecf81b4a71c936c46be => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:22:47.879499 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6c325a-a469-42e5-8142-c54cfec5e8ae None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:47.879795 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6c325a-a469-42e5-8142-c54cfec5e8ae None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:47.880076 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6c325a-a469-42e5-8142-c54cfec5e8ae None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:47.881384 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 68/134] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:22:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:22:47.882228 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6c325a-a469-42e5-8142-c54cfec5e8ae None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:48.008545 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93819a44-a152-4c3e-af04-c02b66a15e19 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:48.071719 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940b3326-5544-4ac9-80e3-5ae9d4db30ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:48.072365 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940b3326-5544-4ac9-80e3-5ae9d4db30ae None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:48.072645 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940b3326-5544-4ac9-80e3-5ae9d4db30ae None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:48.072928 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940b3326-5544-4ac9-80e3-5ae9d4db30ae None admin] PATH_INFO: `/v3/users/8179b7e759db449e9295e696b2333c9c` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:48.074205 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940b3326-5544-4ac9-80e3-5ae9d4db30ae None admin] RBAC: Authorizing `identity:get_user(user_id=8179b7e759db449e9295e696b2333c9c)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:48.076773 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940b3326-5544-4ac9-80e3-5ae9d4db30ae None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:48.078543 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 67/135] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Aug 10 08:22:47 2022] GET /identity/v3/users/8179b7e759db449e9295e696b2333c9c => generated 275 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:22:48.079492 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fdec27-b3e0-4b14-9270-981674146098 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:48.079774 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fdec27-b3e0-4b14-9270-981674146098 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:48.080080 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fdec27-b3e0-4b14-9270-981674146098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:48.091783 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6c325a-a469-42e5-8142-c54cfec5e8ae None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:48.113835 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-58fdec27-b3e0-4b14-9270-981674146098 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:22:48.117071 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-58fdec27-b3e0-4b14-9270-981674146098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:22:48.135018 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b93c26-0ec3-4819-857d-05897a7b434e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:48.135976 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b93c26-0ec3-4819-857d-05897a7b434e None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:48.136339 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b93c26-0ec3-4819-857d-05897a7b434e None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:48.136687 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b93c26-0ec3-4819-857d-05897a7b434e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:48.137542 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b93c26-0ec3-4819-857d-05897a7b434e None admin] RBAC: Adding query filter params (role.id=7237967510e447498d7b37ff7d0ea7ba, scope.project.id=0f2b0269b8b24890b55cca8078954ba1, user.id=3a70f390119d4c168dc285643fd110cc) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:22:48.137912 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b93c26-0ec3-4819-857d-05897a7b434e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:48.139526 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b93c26-0ec3-4819-857d-05897a7b434e None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:48.142952 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:48.147285 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 69/136] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Aug 10 08:22:47 2022] GET /identity/v3/role_assignments?user.id=3a70f390119d4c168dc285643fd110cc&scope.project.id=0f2b0269b8b24890b55cca8078954ba1&role.id=7237967510e447498d7b37ff7d0ea7ba => generated 259 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:22:48.148312 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b93c26-0ec3-4819-857d-05897a7b434e None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:48.232431 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 68/137] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:22:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 08:22:48.269774 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fdec27-b3e0-4b14-9270-981674146098 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:48.300475 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:48.333401 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b93c26-0ec3-4819-857d-05897a7b434e None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:48.420454 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc9b3a7-ef03-45d1-bfa0-579bac1762b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:48.421478 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc9b3a7-ef03-45d1-bfa0-579bac1762b3 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:48.421961 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc9b3a7-ef03-45d1-bfa0-579bac1762b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:48.422615 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc9b3a7-ef03-45d1-bfa0-579bac1762b3 None admin] PATH_INFO: `/v3/projects/fb99099b189242a3bc6159b227bc4e21` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:48.426193 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc9b3a7-ef03-45d1-bfa0-579bac1762b3 None admin] RBAC: Authorizing `identity:get_project(project_id=fb99099b189242a3bc6159b227bc4e21)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:48.427983 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc9b3a7-ef03-45d1-bfa0-579bac1762b3 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:48.435348 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 70/138] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Aug 10 08:22:48 2022] GET /identity/v3/projects/fb99099b189242a3bc6159b227bc4e21 => generated 312 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:22:48.443908 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc9b3a7-ef03-45d1-bfa0-579bac1762b3 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:48.502673 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fdec27-b3e0-4b14-9270-981674146098 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:48.519547 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:48.565354 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66144b0e-44d0-4302-84c0-8fa8258ffc43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:48.565752 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc9b3a7-ef03-45d1-bfa0-579bac1762b3 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:48.565926 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66144b0e-44d0-4302-84c0-8fa8258ffc43 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:48.566182 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66144b0e-44d0-4302-84c0-8fa8258ffc43 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:48.566452 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66144b0e-44d0-4302-84c0-8fa8258ffc43 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:48.567575 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66144b0e-44d0-4302-84c0-8fa8258ffc43 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:48.572974 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66144b0e-44d0-4302-84c0-8fa8258ffc43 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:48.572974 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 69/139] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:22:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:22:48.579648 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66144b0e-44d0-4302-84c0-8fa8258ffc43 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:48.640508 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adbb603-cd3e-4c00-9834-e331320cec49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:48.641201 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adbb603-cd3e-4c00-9834-e331320cec49 None admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:48.641481 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adbb603-cd3e-4c00-9834-e331320cec49 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:48.641756 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adbb603-cd3e-4c00-9834-e331320cec49 None admin] PATH_INFO: `/v3/projects/fb99099b189242a3bc6159b227bc4e21/users/8179b7e759db449e9295e696b2333c9c/roles/44b64db484f24a46bf0b7ced46b519bf` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:48.643708 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adbb603-cd3e-4c00-9834-e331320cec49 None admin] RBAC: Authorizing `identity:create_grant(project_id=fb99099b189242a3bc6159b227bc4e21, user_id=8179b7e759db449e9295e696b2333c9c, role_id=44b64db484f24a46bf0b7ced46b519bf)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:48.645184 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adbb603-cd3e-4c00-9834-e331320cec49 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:48.718239 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 71/140] 10.222.0.57 () {58 vars in 1439 bytes} [Wed Aug 10 08:22:48 2022] PUT /identity/v3/projects/fb99099b189242a3bc6159b227bc4e21/users/8179b7e759db449e9295e696b2333c9c/roles/44b64db484f24a46bf0b7ced46b519bf => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:22:48.719584 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:48.743534 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66144b0e-44d0-4302-84c0-8fa8258ffc43 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:48.792843 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bed8f2f-956e-4dc8-80dd-bf17c9dec887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:48.793485 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bed8f2f-956e-4dc8-80dd-bf17c9dec887 None admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:48.793764 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bed8f2f-956e-4dc8-80dd-bf17c9dec887 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:48.794036 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bed8f2f-956e-4dc8-80dd-bf17c9dec887 None admin] PATH_INFO: `/v3/users` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:48.794700 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bed8f2f-956e-4dc8-80dd-bf17c9dec887 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:48.796348 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bed8f2f-956e-4dc8-80dd-bf17c9dec887 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:48.912805 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 70/141] 10.222.0.57 () {60 vars in 1132 bytes} [Wed Aug 10 08:22:48 2022] POST /identity/v3/users => generated 283 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:22:49.184843 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543cef84-2cf2-41f3-abb0-c52f3cd63838 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:49.185455 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543cef84-2cf2-41f3-abb0-c52f3cd63838 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:49.185455 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543cef84-2cf2-41f3-abb0-c52f3cd63838 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:49.186820 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 72/142] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:22:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:22:49.192470 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded2ea4f-19c2-43c1-a45e-e8d11646547e None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:49.193006 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded2ea4f-19c2-43c1-a45e-e8d11646547e None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:49.193097 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded2ea4f-19c2-43c1-a45e-e8d11646547e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:49.222241 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-ded2ea4f-19c2-43c1-a45e-e8d11646547e None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:22:49.223456 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-ded2ea4f-19c2-43c1-a45e-e8d11646547e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:22:49.320148 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 71/143] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:22:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 08:22:49.359044 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543cef84-2cf2-41f3-abb0-c52f3cd63838 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:49.490401 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543cef84-2cf2-41f3-abb0-c52f3cd63838 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:49.527533 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:49.536257 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44fdf78-67d6-466b-86b2-5968b02a8aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:49.538941 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44fdf78-67d6-466b-86b2-5968b02a8aed None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:49.538941 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44fdf78-67d6-466b-86b2-5968b02a8aed None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:49.538941 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44fdf78-67d6-466b-86b2-5968b02a8aed None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:49.553979 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44fdf78-67d6-466b-86b2-5968b02a8aed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 08:22:49.564818 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-d44fdf78-67d6-466b-86b2-5968b02a8aed None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 08:22:49.564818 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:22:49.564818 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:22:49.564818 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:22:49.564818 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:22:49.564818 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:22:49.564818 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:22:49.564818 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:22:49.564818 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:22:49.564818 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:22:49.564818 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:22:49.564818 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:22:49.564818 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 08:22:49.564818 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 08:22:49.564818 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 08:22:49.564818 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 08:22:49.565779 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 08:22:49.565779 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 08:22:49.565779 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 08:22:49.565779 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 08:22:49.565779 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:22:49.565779 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:22:49.565779 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:22:49.565779 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:22:49.565779 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:22:49.565779 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:22:49.565779 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:22:49.565779 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:22:49.565779 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:22:49.565779 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:22:49.565779 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:22:49.565779 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:22:49.566614 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:22:49.566614 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:22:49.566614 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:22:49.566614 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:22:49.566614 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 08:22:49.566614 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 08:22:49.566614 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 08:22:49.566614 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 08:22:49.566614 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 08:22:49.566614 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 08:22:49.566614 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 08:22:49.566614 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:22:49.566614 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 73/144] 10.222.0.57 () {56 vars in 1114 bytes} [Wed Aug 10 08:22:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:22:49.573786 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded2ea4f-19c2-43c1-a45e-e8d11646547e None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:49.649218 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:49.683547 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded2ea4f-19c2-43c1-a45e-e8d11646547e None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:49.722660 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78d9caa-a03b-4c24-b5a5-1a66ae92917d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:49.723278 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78d9caa-a03b-4c24-b5a5-1a66ae92917d None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:49.723559 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78d9caa-a03b-4c24-b5a5-1a66ae92917d None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:49.723830 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78d9caa-a03b-4c24-b5a5-1a66ae92917d None admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:49.724564 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78d9caa-a03b-4c24-b5a5-1a66ae92917d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:22:49.725321 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78d9caa-a03b-4c24-b5a5-1a66ae92917d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:49.726970 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78d9caa-a03b-4c24-b5a5-1a66ae92917d None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:49.736163 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 72/145] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Aug 10 08:22:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:22:49.741530 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44fdf78-67d6-466b-86b2-5968b02a8aed None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:49.864562 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44fdf78-67d6-466b-86b2-5968b02a8aed None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:49.905567 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7e1c77-e289-430e-99f5-edd145cafa59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:49.906204 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7e1c77-e289-430e-99f5-edd145cafa59 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:49.906530 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7e1c77-e289-430e-99f5-edd145cafa59 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:49.906807 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7e1c77-e289-430e-99f5-edd145cafa59 None admin] PATH_INFO: `/v3/users/8179b7e759db449e9295e696b2333c9c` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:49.907952 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7e1c77-e289-430e-99f5-edd145cafa59 None admin] RBAC: Authorizing `identity:get_user(user_id=8179b7e759db449e9295e696b2333c9c)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:49.909529 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7e1c77-e289-430e-99f5-edd145cafa59 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:49.911121 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 74/146] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Aug 10 08:22:49 2022] GET /identity/v3/users/8179b7e759db449e9295e696b2333c9c => generated 275 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:22:49.917225 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78d9caa-a03b-4c24-b5a5-1a66ae92917d None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:50.024801 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78d9caa-a03b-4c24-b5a5-1a66ae92917d None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:50.058400 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:50.063919 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2978035-f883-44c3-af20-3b15760c77bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:50.064529 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2978035-f883-44c3-af20-3b15760c77bd None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:50.064804 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2978035-f883-44c3-af20-3b15760c77bd None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:50.065129 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2978035-f883-44c3-af20-3b15760c77bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:50.145585 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2978035-f883-44c3-af20-3b15760c77bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:50.147580 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2978035-f883-44c3-af20-3b15760c77bd None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:50.256319 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 73/147] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Aug 10 08:22:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 08:22:50.262642 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7e1c77-e289-430e-99f5-edd145cafa59 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:50.326833 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:50.375688 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7e1c77-e289-430e-99f5-edd145cafa59 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:50.420427 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4d9990-d39e-4a1f-8aee-050231fd7cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:50.421495 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4d9990-d39e-4a1f-8aee-050231fd7cd1 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:50.421974 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4d9990-d39e-4a1f-8aee-050231fd7cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:50.422481 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4d9990-d39e-4a1f-8aee-050231fd7cd1 None admin] PATH_INFO: `/v3/projects/0f2b0269b8b24890b55cca8078954ba1` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:50.423800 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4d9990-d39e-4a1f-8aee-050231fd7cd1 None admin] RBAC: Authorizing `identity:get_project(project_id=0f2b0269b8b24890b55cca8078954ba1)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:50.425525 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4d9990-d39e-4a1f-8aee-050231fd7cd1 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:50.427346 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 75/148] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Aug 10 08:22:50 2022] GET /identity/v3/projects/0f2b0269b8b24890b55cca8078954ba1 => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:22:50.434273 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2978035-f883-44c3-af20-3b15760c77bd None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:50.468597 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:50.552689 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2978035-f883-44c3-af20-3b15760c77bd None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:50.594619 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac67d834-3843-4c29-8b8f-a3104cc6455e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:50.595509 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac67d834-3843-4c29-8b8f-a3104cc6455e None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:50.595979 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac67d834-3843-4c29-8b8f-a3104cc6455e None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:50.596426 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac67d834-3843-4c29-8b8f-a3104cc6455e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:50.597349 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac67d834-3843-4c29-8b8f-a3104cc6455e None admin] RBAC: Adding query filter params (role.id=ccfc8d3fdf2d458299df29001278930f, scope.project.id=0f2b0269b8b24890b55cca8078954ba1, user.id=8179b7e759db449e9295e696b2333c9c) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:22:50.597842 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac67d834-3843-4c29-8b8f-a3104cc6455e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:50.599595 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac67d834-3843-4c29-8b8f-a3104cc6455e None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:50.608006 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 74/149] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Aug 10 08:22:50 2022] GET /identity/v3/role_assignments?user.id=8179b7e759db449e9295e696b2333c9c&scope.project.id=0f2b0269b8b24890b55cca8078954ba1&role.id=ccfc8d3fdf2d458299df29001278930f => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:22:50.634413 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ce998b-596a-4623-a97e-dfb9b7b7ca52 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:50.634986 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ce998b-596a-4623-a97e-dfb9b7b7ca52 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:50.635434 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ce998b-596a-4623-a97e-dfb9b7b7ca52 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:50.637254 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 76/150] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:22:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:22:50.642812 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daae1a2b-df21-4082-81dd-4e2b63d6835b None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:50.643336 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daae1a2b-df21-4082-81dd-4e2b63d6835b None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:50.643790 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daae1a2b-df21-4082-81dd-4e2b63d6835b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:50.667589 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-daae1a2b-df21-4082-81dd-4e2b63d6835b None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:22:50.669154 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-daae1a2b-df21-4082-81dd-4e2b63d6835b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:22:50.762573 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 75/151] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:22:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 08:22:50.809018 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ce998b-596a-4623-a97e-dfb9b7b7ca52 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:50.982856 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ce998b-596a-4623-a97e-dfb9b7b7ca52 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:51.034742 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:51.048876 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8ae330-0b26-4faa-828c-b74bf083c80a None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:51.049314 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8ae330-0b26-4faa-828c-b74bf083c80a None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:51.049571 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8ae330-0b26-4faa-828c-b74bf083c80a None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:51.051179 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 76/152] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:22:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:22:51.057274 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c384301f-ae67-42ea-94a9-75615a4721bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:51.058309 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c384301f-ae67-42ea-94a9-75615a4721bc None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:51.058771 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c384301f-ae67-42ea-94a9-75615a4721bc None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:51.059232 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c384301f-ae67-42ea-94a9-75615a4721bc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:51.068596 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c2b97f-7251-4b74-a951-e972f87fb761 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:51.069063 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c2b97f-7251-4b74-a951-e972f87fb761 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:51.069229 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c2b97f-7251-4b74-a951-e972f87fb761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:51.094435 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c384301f-ae67-42ea-94a9-75615a4721bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 08:22:51.104585 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-c384301f-ae67-42ea-94a9-75615a4721bc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 08:22:51.104585 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:22:51.104585 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:22:51.104585 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:22:51.104585 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:22:51.104585 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:22:51.104585 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:22:51.104585 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:22:51.104585 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:22:51.104585 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:22:51.104585 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:22:51.104585 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:22:51.104585 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 08:22:51.104585 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 08:22:51.104585 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 08:22:51.104585 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 08:22:51.106237 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 08:22:51.106237 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 08:22:51.106237 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 08:22:51.106237 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 08:22:51.106237 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:22:51.106237 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:22:51.106237 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:22:51.106237 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:22:51.106237 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:22:51.106237 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:22:51.106237 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:22:51.106237 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:22:51.106237 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:22:51.106237 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:22:51.106237 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:22:51.106237 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:22:51.107708 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:22:51.107708 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:22:51.107708 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:22:51.107708 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:22:51.107708 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 08:22:51.107708 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 08:22:51.107708 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 08:22:51.107708 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 08:22:51.107708 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 08:22:51.107708 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 08:22:51.107708 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 08:22:51.107708 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:22:51.108833 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-74c2b97f-7251-4b74-a951-e972f87fb761 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:22:51.108833 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-74c2b97f-7251-4b74-a951-e972f87fb761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:22:51.110449 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 77/153] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Aug 10 08:22:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:22:51.119860 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c384301f-ae67-42ea-94a9-75615a4721bc None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:51.151954 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:51.294617 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 77/154] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:22:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 08:22:51.295632 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9012e9c5-7626-46be-91d8-63dab02eb82e None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:51.295914 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9012e9c5-7626-46be-91d8-63dab02eb82e None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:51.296184 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9012e9c5-7626-46be-91d8-63dab02eb82e None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:51.297518 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 78/155] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:22:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:22:51.306149 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a162a2-848e-41a3-ab82-b5179fe23fe9 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:51.306512 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a162a2-848e-41a3-ab82-b5179fe23fe9 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:51.306730 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a162a2-848e-41a3-ab82-b5179fe23fe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:51.342010 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-39a162a2-848e-41a3-ab82-b5179fe23fe9 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:22:51.344236 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-39a162a2-848e-41a3-ab82-b5179fe23fe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:22:51.401589 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c384301f-ae67-42ea-94a9-75615a4721bc None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:51.460298 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 79/156] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:22:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 08:22:51.461255 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a162a2-848e-41a3-ab82-b5179fe23fe9 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:51.550302 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846c6191-7013-423a-8704-27ff52ff5e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:51.550847 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846c6191-7013-423a-8704-27ff52ff5e01 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:51.551136 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846c6191-7013-423a-8704-27ff52ff5e01 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:51.551408 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846c6191-7013-423a-8704-27ff52ff5e01 None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:51.552087 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846c6191-7013-423a-8704-27ff52ff5e01 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:22:51.552393 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846c6191-7013-423a-8704-27ff52ff5e01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:51.554443 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846c6191-7013-423a-8704-27ff52ff5e01 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:51.571681 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 78/157] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Aug 10 08:22:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 453 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:22:51.572690 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846c6191-7013-423a-8704-27ff52ff5e01 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:51.577311 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:51.611889 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a162a2-848e-41a3-ab82-b5179fe23fe9 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:51.626060 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:51.687059 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846c6191-7013-423a-8704-27ff52ff5e01 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:51.717610 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd0a1c2-0097-4e49-965f-ca36c5686c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:51.718618 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd0a1c2-0097-4e49-965f-ca36c5686c17 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:51.719103 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd0a1c2-0097-4e49-965f-ca36c5686c17 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:51.719579 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd0a1c2-0097-4e49-965f-ca36c5686c17 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:51.732960 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36821c5b-e368-49ba-ad24-e98688c7cd27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:51.733905 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36821c5b-e368-49ba-ad24-e98688c7cd27 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:51.734408 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36821c5b-e368-49ba-ad24-e98688c7cd27 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:51.734892 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36821c5b-e368-49ba-ad24-e98688c7cd27 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:51.736372 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd0a1c2-0097-4e49-965f-ca36c5686c17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 08:22:51.742136 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-bcd0a1c2-0097-4e49-965f-ca36c5686c17 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 08:22:51.742136 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:22:51.742136 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:22:51.742136 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:22:51.742136 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:22:51.742136 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:22:51.742136 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:22:51.742136 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:22:51.742136 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:22:51.742136 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:22:51.742136 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:22:51.742136 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:22:51.742136 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 08:22:51.742136 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 08:22:51.742136 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 08:22:51.742136 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 08:22:51.743034 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 08:22:51.743034 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 08:22:51.743034 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 08:22:51.743034 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 08:22:51.743034 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:22:51.743034 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:22:51.743034 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:22:51.743034 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:22:51.743034 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:22:51.743034 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:22:51.743034 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:22:51.743034 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:22:51.743034 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:22:51.743034 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:22:51.743034 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:22:51.743034 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:22:51.743815 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:22:51.743815 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:22:51.743815 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:22:51.743815 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:22:51.743815 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 08:22:51.743815 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 08:22:51.743815 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 08:22:51.743815 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 08:22:51.743815 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 08:22:51.743815 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 08:22:51.743815 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 08:22:51.743815 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:22:51.750506 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 80/158] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Aug 10 08:22:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:22:51.751883 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd0a1c2-0097-4e49-965f-ca36c5686c17 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:51.767462 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36821c5b-e368-49ba-ad24-e98688c7cd27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 08:22:51.773554 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-36821c5b-e368-49ba-ad24-e98688c7cd27 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 08:22:51.773554 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:22:51.773554 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:22:51.773554 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:22:51.773554 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:22:51.773554 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:22:51.773554 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:22:51.773554 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:22:51.773554 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:22:51.773554 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:22:51.773554 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:22:51.773554 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:22:51.773554 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 08:22:51.773554 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 08:22:51.773554 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 08:22:51.773554 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 08:22:51.774421 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 08:22:51.774421 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 08:22:51.774421 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 08:22:51.774421 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 08:22:51.774421 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:22:51.774421 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:22:51.774421 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:22:51.774421 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:22:51.774421 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:22:51.774421 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:22:51.774421 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:22:51.774421 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:22:51.774421 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:22:51.774421 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:22:51.774421 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:22:51.774421 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:22:51.775271 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:22:51.775271 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:22:51.775271 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:22:51.775271 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:22:51.775271 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 08:22:51.775271 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 08:22:51.775271 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 08:22:51.775271 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 08:22:51.775271 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 08:22:51.775271 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 08:22:51.775271 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 08:22:51.775271 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:22:51.785240 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 79/159] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Aug 10 08:22:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:22:51.786326 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36821c5b-e368-49ba-ad24-e98688c7cd27 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:51.942858 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd0a1c2-0097-4e49-965f-ca36c5686c17 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:51.986165 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36821c5b-e368-49ba-ad24-e98688c7cd27 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:52.002927 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:52.038862 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e01e6d-ddef-4ebd-b55c-aed4a79ab0b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:52.039522 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e01e6d-ddef-4ebd-b55c-aed4a79ab0b5 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:52.039804 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e01e6d-ddef-4ebd-b55c-aed4a79ab0b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:52.040080 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e01e6d-ddef-4ebd-b55c-aed4a79ab0b5 None admin] PATH_INFO: `/v3/users/a79bffd3ad024a2f938d1f77cfd728d5` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:52.045702 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e01e6d-ddef-4ebd-b55c-aed4a79ab0b5 None admin] RBAC: Authorizing `identity:get_user(user_id=a79bffd3ad024a2f938d1f77cfd728d5)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:52.047183 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e01e6d-ddef-4ebd-b55c-aed4a79ab0b5 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:52.049086 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 81/160] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Aug 10 08:22:51 2022] GET /identity/v3/users/a79bffd3ad024a2f938d1f77cfd728d5 => generated 247 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:22:52.049790 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e01e6d-ddef-4ebd-b55c-aed4a79ab0b5 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:52.054404 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b68310-821c-4f12-96d7-9c5b9f738782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:52.055108 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b68310-821c-4f12-96d7-9c5b9f738782 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:52.055313 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b68310-821c-4f12-96d7-9c5b9f738782 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:52.055581 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b68310-821c-4f12-96d7-9c5b9f738782 None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:52.056258 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b68310-821c-4f12-96d7-9c5b9f738782 None admin] RBAC: Adding query filter params (name=member) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:22:52.056559 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b68310-821c-4f12-96d7-9c5b9f738782 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:52.058510 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b68310-821c-4f12-96d7-9c5b9f738782 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:52.074985 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 80/161] 10.222.0.57 () {56 vars in 1101 bytes} [Wed Aug 10 08:22:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:22:52.075693 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b68310-821c-4f12-96d7-9c5b9f738782 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:52.089930 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:52.283697 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b68310-821c-4f12-96d7-9c5b9f738782 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:52.287031 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e01e6d-ddef-4ebd-b55c-aed4a79ab0b5 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:52.352080 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a5ea8c-05bf-429b-8e0a-68cdcc30d830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:52.353525 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a5ea8c-05bf-429b-8e0a-68cdcc30d830 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:52.354023 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a5ea8c-05bf-429b-8e0a-68cdcc30d830 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:52.354509 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a5ea8c-05bf-429b-8e0a-68cdcc30d830 None admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:52.355376 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a5ea8c-05bf-429b-8e0a-68cdcc30d830 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:22:52.355872 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a5ea8c-05bf-429b-8e0a-68cdcc30d830 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:52.357498 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a5ea8c-05bf-429b-8e0a-68cdcc30d830 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:52.358270 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc77d3c-d2bd-4b45-88dc-a4f61c0dbfa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:52.363861 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc77d3c-d2bd-4b45-88dc-a4f61c0dbfa9 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:52.363861 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc77d3c-d2bd-4b45-88dc-a4f61c0dbfa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:52.363861 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc77d3c-d2bd-4b45-88dc-a4f61c0dbfa9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:52.372689 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 82/162] 10.222.0.57 () {56 vars in 1101 bytes} [Wed Aug 10 08:22:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:22:52.373826 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a5ea8c-05bf-429b-8e0a-68cdcc30d830 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:52.442808 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:52.518988 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc77d3c-d2bd-4b45-88dc-a4f61c0dbfa9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:52.520603 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc77d3c-d2bd-4b45-88dc-a4f61c0dbfa9 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:52.527494 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:52.579903 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a5ea8c-05bf-429b-8e0a-68cdcc30d830 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:52.672098 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a64035-16ed-4ad4-990d-a7cb9ac04fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:52.672738 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a64035-16ed-4ad4-990d-a7cb9ac04fc9 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:52.673054 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a64035-16ed-4ad4-990d-a7cb9ac04fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:52.673342 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a64035-16ed-4ad4-990d-a7cb9ac04fc9 None admin] PATH_INFO: `/v3/users/8179b7e759db449e9295e696b2333c9c` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:52.676053 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a64035-16ed-4ad4-990d-a7cb9ac04fc9 None admin] RBAC: Authorizing `identity:get_user(user_id=8179b7e759db449e9295e696b2333c9c)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:52.677770 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a64035-16ed-4ad4-990d-a7cb9ac04fc9 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:52.679584 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 83/163] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Aug 10 08:22:52 2022] GET /identity/v3/users/8179b7e759db449e9295e696b2333c9c => generated 275 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:22:52.680180 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a64035-16ed-4ad4-990d-a7cb9ac04fc9 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:52.773306 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 81/164] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Aug 10 08:22:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 698 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 08:22:52.774373 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc77d3c-d2bd-4b45-88dc-a4f61c0dbfa9 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:52.867726 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a64035-16ed-4ad4-990d-a7cb9ac04fc9 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:52.929307 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:52.931959 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c663dd86-9163-4a7e-8c0f-b97c93b7f29b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:52.932785 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c663dd86-9163-4a7e-8c0f-b97c93b7f29b None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:52.933205 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c663dd86-9163-4a7e-8c0f-b97c93b7f29b None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:52.933719 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c663dd86-9163-4a7e-8c0f-b97c93b7f29b None admin] PATH_INFO: `/v3/users/3a70f390119d4c168dc285643fd110cc` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:52.939777 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c663dd86-9163-4a7e-8c0f-b97c93b7f29b None admin] RBAC: Authorizing `identity:get_user(user_id=3a70f390119d4c168dc285643fd110cc)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:52.941479 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c663dd86-9163-4a7e-8c0f-b97c93b7f29b None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:52.943334 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 84/165] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Aug 10 08:22:52 2022] GET /identity/v3/users/3a70f390119d4c168dc285643fd110cc => generated 289 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:22:52.944633 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb10cbc3-c460-4869-90d3-c02e369e712e None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:52.947344 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb10cbc3-c460-4869-90d3-c02e369e712e None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:52.947695 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb10cbc3-c460-4869-90d3-c02e369e712e None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:52.949004 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 85/166] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:22:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:22:52.949717 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb10cbc3-c460-4869-90d3-c02e369e712e None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:52.986974 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:52.997904 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc77d3c-d2bd-4b45-88dc-a4f61c0dbfa9 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:53.119806 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de2d025-fd31-484b-adc0-a9b9efc7ee38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:53.123100 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de2d025-fd31-484b-adc0-a9b9efc7ee38 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:53.123100 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de2d025-fd31-484b-adc0-a9b9efc7ee38 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:53.123100 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de2d025-fd31-484b-adc0-a9b9efc7ee38 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:53.123353 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb10cbc3-c460-4869-90d3-c02e369e712e None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:53.167421 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6a44cf-6220-4ca2-b2a5-6f6ffef7cdae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:53.168053 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6a44cf-6220-4ca2-b2a5-6f6ffef7cdae None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:53.168331 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6a44cf-6220-4ca2-b2a5-6f6ffef7cdae None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:53.168599 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6a44cf-6220-4ca2-b2a5-6f6ffef7cdae None admin] PATH_INFO: `/v3/projects/0f2b0269b8b24890b55cca8078954ba1` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:53.170009 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6a44cf-6220-4ca2-b2a5-6f6ffef7cdae None admin] RBAC: Authorizing `identity:get_project(project_id=0f2b0269b8b24890b55cca8078954ba1)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:53.172365 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6a44cf-6220-4ca2-b2a5-6f6ffef7cdae None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:53.174372 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 86/167] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Aug 10 08:22:52 2022] GET /identity/v3/projects/0f2b0269b8b24890b55cca8078954ba1 => generated 298 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:22:53.175148 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6a44cf-6220-4ca2-b2a5-6f6ffef7cdae None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:53.249342 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de2d025-fd31-484b-adc0-a9b9efc7ee38 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:53.251035 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de2d025-fd31-484b-adc0-a9b9efc7ee38 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:53.287355 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:53.310922 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6a44cf-6220-4ca2-b2a5-6f6ffef7cdae None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:53.354721 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63937acf-1b4c-40df-88d7-eea451599c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:53.355454 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63937acf-1b4c-40df-88d7-eea451599c54 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:53.355683 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63937acf-1b4c-40df-88d7-eea451599c54 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:53.355957 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63937acf-1b4c-40df-88d7-eea451599c54 None admin] PATH_INFO: `/v3/projects/0f2b0269b8b24890b55cca8078954ba1` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:53.357388 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63937acf-1b4c-40df-88d7-eea451599c54 None admin] RBAC: Authorizing `identity:get_project(project_id=0f2b0269b8b24890b55cca8078954ba1)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:53.360104 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63937acf-1b4c-40df-88d7-eea451599c54 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:53.362681 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 87/168] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Aug 10 08:22:53 2022] GET /identity/v3/projects/0f2b0269b8b24890b55cca8078954ba1 => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:22:53.364154 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb948f6c-7a2f-41f5-8944-cbf4a31bf283 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:53.364438 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb948f6c-7a2f-41f5-8944-cbf4a31bf283 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:53.364711 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb948f6c-7a2f-41f5-8944-cbf4a31bf283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:53.372654 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:53.398682 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-eb948f6c-7a2f-41f5-8944-cbf4a31bf283 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:22:53.400247 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-eb948f6c-7a2f-41f5-8944-cbf4a31bf283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:22:53.431788 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 82/169] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Aug 10 08:22:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 658 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 08:22:53.433837 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d4d99e-b6d2-49fe-86b4-7c40e53c4db3 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:53.434332 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d4d99e-b6d2-49fe-86b4-7c40e53c4db3 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:53.434784 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d4d99e-b6d2-49fe-86b4-7c40e53c4db3 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:53.440537 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 83/170] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:22:53 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:22:53.441575 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d4d99e-b6d2-49fe-86b4-7c40e53c4db3 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:53.496936 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 88/171] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:22:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 08:22:53.498072 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb948f6c-7a2f-41f5-8944-cbf4a31bf283 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:53.561860 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d4d99e-b6d2-49fe-86b4-7c40e53c4db3 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:53.608096 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:53.610081 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb4d008-fbf6-483e-bd48-cdd652faa56d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:53.610690 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb4d008-fbf6-483e-bd48-cdd652faa56d None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:53.610973 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb4d008-fbf6-483e-bd48-cdd652faa56d None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:53.611243 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb4d008-fbf6-483e-bd48-cdd652faa56d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:53.611973 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb4d008-fbf6-483e-bd48-cdd652faa56d None admin] RBAC: Adding query filter params (role.id=ec3751692d9c4ecf81b4a71c936c46be, scope.project.id=0f2b0269b8b24890b55cca8078954ba1, user.id=a79bffd3ad024a2f938d1f77cfd728d5) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:22:53.612278 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb4d008-fbf6-483e-bd48-cdd652faa56d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:53.613763 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb4d008-fbf6-483e-bd48-cdd652faa56d None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:53.620384 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 84/172] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Aug 10 08:22:53 2022] GET /identity/v3/role_assignments?user.id=a79bffd3ad024a2f938d1f77cfd728d5&scope.project.id=0f2b0269b8b24890b55cca8078954ba1&role.id=ec3751692d9c4ecf81b4a71c936c46be => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:22:53.622036 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02912f15-f562-4610-91a1-1a60d35f6fbd None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:53.622324 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02912f15-f562-4610-91a1-1a60d35f6fbd None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:53.622598 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02912f15-f562-4610-91a1-1a60d35f6fbd None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:53.623997 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 85/173] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:22:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:22:53.624714 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02912f15-f562-4610-91a1-1a60d35f6fbd None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:53.655598 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb948f6c-7a2f-41f5-8944-cbf4a31bf283 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:53.676209 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:53.695830 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb73bd44-f40b-4812-b198-7519cb54fa31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:53.696612 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb73bd44-f40b-4812-b198-7519cb54fa31 None admin] REQUEST_METHOD: `PUT` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:53.696896 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb73bd44-f40b-4812-b198-7519cb54fa31 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:53.697196 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb73bd44-f40b-4812-b198-7519cb54fa31 None admin] PATH_INFO: `/v3/projects/0f2b0269b8b24890b55cca8078954ba1/users/3a70f390119d4c168dc285643fd110cc/roles/7237967510e447498d7b37ff7d0ea7ba` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:53.699205 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb73bd44-f40b-4812-b198-7519cb54fa31 None admin] RBAC: Authorizing `identity:create_grant(project_id=0f2b0269b8b24890b55cca8078954ba1, user_id=3a70f390119d4c168dc285643fd110cc, role_id=7237967510e447498d7b37ff7d0ea7ba)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:53.700696 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb73bd44-f40b-4812-b198-7519cb54fa31 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:53.755535 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02912f15-f562-4610-91a1-1a60d35f6fbd None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:53.765232 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 89/174] 10.222.0.57 () {58 vars in 1439 bytes} [Wed Aug 10 08:22:53 2022] PUT /identity/v3/projects/0f2b0269b8b24890b55cca8078954ba1/users/3a70f390119d4c168dc285643fd110cc/roles/7237967510e447498d7b37ff7d0ea7ba => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:22:53.766823 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfaafc9-fd59-46ba-8a6b-223ea1532246 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:53.767271 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfaafc9-fd59-46ba-8a6b-223ea1532246 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:53.767700 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfaafc9-fd59-46ba-8a6b-223ea1532246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:53.794243 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-abfaafc9-fd59-46ba-8a6b-223ea1532246 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:22:53.795997 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-abfaafc9-fd59-46ba-8a6b-223ea1532246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:22:53.815734 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa56538-41dd-4421-946a-98c756656eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:53.816385 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa56538-41dd-4421-946a-98c756656eb8 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:53.816683 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa56538-41dd-4421-946a-98c756656eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:53.816978 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa56538-41dd-4421-946a-98c756656eb8 None admin] PATH_INFO: `/v3/projects/0f2b0269b8b24890b55cca8078954ba1` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:53.818147 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa56538-41dd-4421-946a-98c756656eb8 None admin] RBAC: Authorizing `identity:get_project(project_id=0f2b0269b8b24890b55cca8078954ba1)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:53.819885 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa56538-41dd-4421-946a-98c756656eb8 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:53.821713 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 86/175] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Aug 10 08:22:53 2022] GET /identity/v3/projects/0f2b0269b8b24890b55cca8078954ba1 => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:22:53.822545 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa56538-41dd-4421-946a-98c756656eb8 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:53.894887 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 90/176] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:22:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 08:22:53.896393 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb6bce4-04fd-4c74-b41c-c3e4a8cf74a4 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:53.896791 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb6bce4-04fd-4c74-b41c-c3e4a8cf74a4 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:53.897764 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb6bce4-04fd-4c74-b41c-c3e4a8cf74a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:53.924452 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-bfb6bce4-04fd-4c74-b41c-c3e4a8cf74a4 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:22:53.925709 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-bfb6bce4-04fd-4c74-b41c-c3e4a8cf74a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:22:53.956811 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:53.962073 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa56538-41dd-4421-946a-98c756656eb8 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:53.995019 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:53.998051 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020a3d30-371a-4e66-bc52-c38a439fc3a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:53.998667 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020a3d30-371a-4e66-bc52-c38a439fc3a7 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:53.998989 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020a3d30-371a-4e66-bc52-c38a439fc3a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:53.999237 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020a3d30-371a-4e66-bc52-c38a439fc3a7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:54.000853 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 91/177] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:22:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 08:22:54.001937 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdf2f9e-de11-4741-af28-4ded2fd9bc08 None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:54.002271 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdf2f9e-de11-4741-af28-4ded2fd9bc08 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:54.002537 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdf2f9e-de11-4741-af28-4ded2fd9bc08 None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:54.003809 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 92/178] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:22:54 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:22:54.008362 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdf2f9e-de11-4741-af28-4ded2fd9bc08 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:54.024374 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020a3d30-371a-4e66-bc52-c38a439fc3a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 08:22:54.029976 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-020a3d30-371a-4e66-bc52-c38a439fc3a7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 08:22:54.029976 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:22:54.029976 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:22:54.029976 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:22:54.029976 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:22:54.029976 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:22:54.029976 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:22:54.029976 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:22:54.029976 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:22:54.029976 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:22:54.029976 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:22:54.029976 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:22:54.029976 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 08:22:54.029976 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 08:22:54.029976 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 08:22:54.029976 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 08:22:54.030906 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 08:22:54.030906 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 08:22:54.030906 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 08:22:54.030906 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 08:22:54.030906 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:22:54.030906 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:22:54.030906 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:22:54.030906 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:22:54.030906 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:22:54.030906 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:22:54.030906 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:22:54.030906 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:22:54.030906 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:22:54.030906 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:22:54.030906 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:22:54.030906 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:22:54.031704 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:22:54.031704 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:22:54.031704 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:22:54.031704 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:22:54.031704 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 08:22:54.031704 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 08:22:54.031704 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 08:22:54.031704 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 08:22:54.031704 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 08:22:54.031704 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 08:22:54.031704 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 08:22:54.031704 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:22:54.031704 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 87/179] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Aug 10 08:22:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:22:54.032368 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020a3d30-371a-4e66-bc52-c38a439fc3a7 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:54.121388 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdf2f9e-de11-4741-af28-4ded2fd9bc08 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:54.149201 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020a3d30-371a-4e66-bc52-c38a439fc3a7 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:54.164109 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b497cec1-310f-4b9c-aaf1-ffa4d586132a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:54.164720 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b497cec1-310f-4b9c-aaf1-ffa4d586132a None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:54.165022 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b497cec1-310f-4b9c-aaf1-ffa4d586132a None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:54.165293 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b497cec1-310f-4b9c-aaf1-ffa4d586132a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:54.166032 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b497cec1-310f-4b9c-aaf1-ffa4d586132a None admin] RBAC: Adding query filter params (role.id=44b64db484f24a46bf0b7ced46b519bf, scope.project.id=0f2b0269b8b24890b55cca8078954ba1, user.id=8179b7e759db449e9295e696b2333c9c) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:22:54.166335 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b497cec1-310f-4b9c-aaf1-ffa4d586132a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:54.167784 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b497cec1-310f-4b9c-aaf1-ffa4d586132a None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:54.173865 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 93/180] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Aug 10 08:22:54 2022] GET /identity/v3/role_assignments?user.id=8179b7e759db449e9295e696b2333c9c&scope.project.id=0f2b0269b8b24890b55cca8078954ba1&role.id=44b64db484f24a46bf0b7ced46b519bf => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:22:54.183215 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b8c19b-000b-417d-8ee6-7684e67b7b71 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:54.183572 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b8c19b-000b-417d-8ee6-7684e67b7b71 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:54.183771 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b8c19b-000b-417d-8ee6-7684e67b7b71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:54.190820 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4c3e32-1f5d-4c84-9fa4-2a4e1f6835ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:54.191863 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4c3e32-1f5d-4c84-9fa4-2a4e1f6835ce None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:54.192344 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4c3e32-1f5d-4c84-9fa4-2a4e1f6835ce None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:54.192834 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4c3e32-1f5d-4c84-9fa4-2a4e1f6835ce None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:54.207424 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4c3e32-1f5d-4c84-9fa4-2a4e1f6835ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 08:22:54.208063 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-29b8c19b-000b-417d-8ee6-7684e67b7b71 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:22:54.209332 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-29b8c19b-000b-417d-8ee6-7684e67b7b71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:22:54.214566 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-2a4c3e32-1f5d-4c84-9fa4-2a4e1f6835ce None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 08:22:54.214566 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:22:54.214566 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:22:54.214566 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:22:54.214566 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:22:54.214566 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:22:54.214566 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:22:54.214566 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:22:54.214566 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:22:54.214566 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:22:54.214566 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:22:54.214566 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:22:54.214566 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 08:22:54.214566 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 08:22:54.214566 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 08:22:54.214566 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 08:22:54.215551 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 08:22:54.215551 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 08:22:54.215551 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 08:22:54.215551 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 08:22:54.215551 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:22:54.215551 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:22:54.215551 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:22:54.215551 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:22:54.215551 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:22:54.215551 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:22:54.215551 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:22:54.215551 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:22:54.215551 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:22:54.215551 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:22:54.215551 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:22:54.215551 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:22:54.216344 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:22:54.216344 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:22:54.216344 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:22:54.216344 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:22:54.216344 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 08:22:54.216344 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 08:22:54.216344 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 08:22:54.216344 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 08:22:54.216344 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 08:22:54.216344 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 08:22:54.216344 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 08:22:54.216344 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:22:54.218927 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 88/181] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Aug 10 08:22:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:22:54.220099 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4c3e32-1f5d-4c84-9fa4-2a4e1f6835ce None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:54.251013 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:54.287628 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:54.304883 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 94/182] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:22:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 08:22:54.304883 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b8c19b-000b-417d-8ee6-7684e67b7b71 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:54.390877 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4c3e32-1f5d-4c84-9fa4-2a4e1f6835ce None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:54.437653 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b8c19b-000b-417d-8ee6-7684e67b7b71 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:54.440432 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab89491c-ed2f-48f8-9ecf-4524c36b4387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:54.441667 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab89491c-ed2f-48f8-9ecf-4524c36b4387 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:54.441667 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab89491c-ed2f-48f8-9ecf-4524c36b4387 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:54.441667 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab89491c-ed2f-48f8-9ecf-4524c36b4387 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:54.446015 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab89491c-ed2f-48f8-9ecf-4524c36b4387 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:54.447069 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab89491c-ed2f-48f8-9ecf-4524c36b4387 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:54.448842 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 89/183] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:22:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:22:54.450213 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab89491c-ed2f-48f8-9ecf-4524c36b4387 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:54.486836 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5725b0a2-226b-4ca7-a242-67c379760dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:54.487547 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5725b0a2-226b-4ca7-a242-67c379760dc2 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:54.487776 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5725b0a2-226b-4ca7-a242-67c379760dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:54.488056 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5725b0a2-226b-4ca7-a242-67c379760dc2 None admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:54.488952 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5725b0a2-226b-4ca7-a242-67c379760dc2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:22:54.489084 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5725b0a2-226b-4ca7-a242-67c379760dc2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:54.490618 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5725b0a2-226b-4ca7-a242-67c379760dc2 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:54.499977 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 95/184] 10.222.0.57 () {56 vars in 1101 bytes} [Wed Aug 10 08:22:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:22:54.500771 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5725b0a2-226b-4ca7-a242-67c379760dc2 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:54.559419 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:54.562124 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:54.571894 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab89491c-ed2f-48f8-9ecf-4524c36b4387 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:54.606063 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5725b0a2-226b-4ca7-a242-67c379760dc2 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:54.609640 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4305dd8d-314d-499a-a1fd-5d723d861487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:54.610264 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4305dd8d-314d-499a-a1fd-5d723d861487 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:54.610528 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4305dd8d-314d-499a-a1fd-5d723d861487 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:54.610798 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4305dd8d-314d-499a-a1fd-5d723d861487 None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:54.611464 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4305dd8d-314d-499a-a1fd-5d723d861487 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:22:54.611767 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4305dd8d-314d-499a-a1fd-5d723d861487 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:54.613572 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4305dd8d-314d-499a-a1fd-5d723d861487 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:54.629157 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 90/185] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Aug 10 08:22:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:22:54.629157 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4305dd8d-314d-499a-a1fd-5d723d861487 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:54.641430 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0aa67b-e91f-4bfd-95b3-61d469231727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:54.642060 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0aa67b-e91f-4bfd-95b3-61d469231727 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:54.642345 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0aa67b-e91f-4bfd-95b3-61d469231727 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:54.642603 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0aa67b-e91f-4bfd-95b3-61d469231727 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:54.652960 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0aa67b-e91f-4bfd-95b3-61d469231727 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 08:22:54.658705 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-8b0aa67b-e91f-4bfd-95b3-61d469231727 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 08:22:54.658705 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:22:54.658705 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:22:54.658705 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:22:54.658705 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:22:54.658705 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:22:54.658705 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:22:54.658705 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:22:54.658705 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:22:54.658705 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:22:54.658705 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:22:54.658705 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:22:54.658705 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 08:22:54.658705 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 08:22:54.658705 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 08:22:54.658705 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 08:22:54.659602 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 08:22:54.659602 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 08:22:54.659602 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 08:22:54.659602 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 08:22:54.659602 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:22:54.659602 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:22:54.659602 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:22:54.659602 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:22:54.659602 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:22:54.659602 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:22:54.659602 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:22:54.659602 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:22:54.659602 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:22:54.659602 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:22:54.659602 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:22:54.659602 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:22:54.660442 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:22:54.660442 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:22:54.660442 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:22:54.660442 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:22:54.660442 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 08:22:54.660442 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 08:22:54.660442 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 08:22:54.660442 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 08:22:54.660442 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 08:22:54.660442 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 08:22:54.660442 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 08:22:54.660442 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:22:54.660442 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 96/186] 10.222.0.57 () {56 vars in 1114 bytes} [Wed Aug 10 08:22:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:22:54.661226 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0aa67b-e91f-4bfd-95b3-61d469231727 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:54.740142 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4305dd8d-314d-499a-a1fd-5d723d861487 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:54.769829 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0aa67b-e91f-4bfd-95b3-61d469231727 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:54.775173 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03e2ece-ef3c-4abc-b3fb-b505475c03dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:54.775790 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03e2ece-ef3c-4abc-b3fb-b505475c03dd None admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:54.776072 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03e2ece-ef3c-4abc-b3fb-b505475c03dd None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:54.776344 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03e2ece-ef3c-4abc-b3fb-b505475c03dd None admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:54.777015 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03e2ece-ef3c-4abc-b3fb-b505475c03dd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:54.778455 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03e2ece-ef3c-4abc-b3fb-b505475c03dd None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:54.785844 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:54.807220 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba773fa-a61f-4787-8d89-b0735413ccf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:54.807932 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba773fa-a61f-4787-8d89-b0735413ccf3 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:54.808294 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba773fa-a61f-4787-8d89-b0735413ccf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:54.808618 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba773fa-a61f-4787-8d89-b0735413ccf3 None admin] PATH_INFO: `/v3/users/8179b7e759db449e9295e696b2333c9c` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:54.809877 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba773fa-a61f-4787-8d89-b0735413ccf3 None admin] RBAC: Authorizing `identity:get_user(user_id=8179b7e759db449e9295e696b2333c9c)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:54.811344 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba773fa-a61f-4787-8d89-b0735413ccf3 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:54.812955 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 97/187] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Aug 10 08:22:54 2022] GET /identity/v3/users/8179b7e759db449e9295e696b2333c9c => generated 275 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:22:54.813724 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba773fa-a61f-4787-8d89-b0735413ccf3 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:54.896362 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 91/188] 10.222.0.57 () {60 vars in 1132 bytes} [Wed Aug 10 08:22:54 2022] POST /identity/v3/users => generated 297 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:22:54.906032 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03e2ece-ef3c-4abc-b3fb-b505475c03dd None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:54.924048 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:54.929753 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba773fa-a61f-4787-8d89-b0735413ccf3 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:54.972717 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029294cc-e1eb-450e-8a39-3ba6e2e554d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:54.973718 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029294cc-e1eb-450e-8a39-3ba6e2e554d3 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:54.974240 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029294cc-e1eb-450e-8a39-3ba6e2e554d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:54.974800 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029294cc-e1eb-450e-8a39-3ba6e2e554d3 None admin] PATH_INFO: `/v3/users/e547c2d8da97403c8938e5a58c1003b1` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:54.991570 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029294cc-e1eb-450e-8a39-3ba6e2e554d3 None admin] RBAC: Authorizing `identity:get_user(user_id=e547c2d8da97403c8938e5a58c1003b1)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:54.994696 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029294cc-e1eb-450e-8a39-3ba6e2e554d3 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:54.996962 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 98/189] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Aug 10 08:22:54 2022] GET /identity/v3/users/e547c2d8da97403c8938e5a58c1003b1 => generated 283 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:22:55.002052 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029294cc-e1eb-450e-8a39-3ba6e2e554d3 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:55.053008 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:55.070741 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03e2ece-ef3c-4abc-b3fb-b505475c03dd None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:55.113651 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dd2f92-7e94-4609-aadf-5ccdb243a9cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:55.114610 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dd2f92-7e94-4609-aadf-5ccdb243a9cf None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:55.115067 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dd2f92-7e94-4609-aadf-5ccdb243a9cf None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:55.115720 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dd2f92-7e94-4609-aadf-5ccdb243a9cf None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:55.118339 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dd2f92-7e94-4609-aadf-5ccdb243a9cf None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:22:55.118927 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dd2f92-7e94-4609-aadf-5ccdb243a9cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:55.120340 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dd2f92-7e94-4609-aadf-5ccdb243a9cf None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:55.132520 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 92/190] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Aug 10 08:22:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:22:55.133396 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029294cc-e1eb-450e-8a39-3ba6e2e554d3 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:55.133591 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dd2f92-7e94-4609-aadf-5ccdb243a9cf None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:55.172961 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3612905e-07e1-4441-b59b-e717314c1b3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:55.173830 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3612905e-07e1-4441-b59b-e717314c1b3d None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:55.174255 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3612905e-07e1-4441-b59b-e717314c1b3d None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:55.174673 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3612905e-07e1-4441-b59b-e717314c1b3d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:55.216801 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:55.259035 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dd2f92-7e94-4609-aadf-5ccdb243a9cf None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:55.266774 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3612905e-07e1-4441-b59b-e717314c1b3d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:55.268806 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3612905e-07e1-4441-b59b-e717314c1b3d None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:55.302956 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654c709c-a73e-4fc1-a681-c46733e6c171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:55.304334 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654c709c-a73e-4fc1-a681-c46733e6c171 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:55.304619 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654c709c-a73e-4fc1-a681-c46733e6c171 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:55.305027 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654c709c-a73e-4fc1-a681-c46733e6c171 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:55.322266 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:55.398863 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654c709c-a73e-4fc1-a681-c46733e6c171 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:55.400364 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654c709c-a73e-4fc1-a681-c46733e6c171 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:55.411535 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 99/191] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Aug 10 08:22:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 08:22:55.412391 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3612905e-07e1-4441-b59b-e717314c1b3d None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:55.476038 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:55.507396 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 93/192] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Aug 10 08:22:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 08:22:55.510419 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654c709c-a73e-4fc1-a681-c46733e6c171 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:55.614688 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654c709c-a73e-4fc1-a681-c46733e6c171 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:55.615753 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3612905e-07e1-4441-b59b-e717314c1b3d None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:55.652988 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d131a6-d07b-4e1d-bf23-1083d20bab18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:55.652988 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d131a6-d07b-4e1d-bf23-1083d20bab18 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:55.652988 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d131a6-d07b-4e1d-bf23-1083d20bab18 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:55.652988 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d131a6-d07b-4e1d-bf23-1083d20bab18 None admin] PATH_INFO: `/v3/projects/fb99099b189242a3bc6159b227bc4e21` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:55.654116 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d131a6-d07b-4e1d-bf23-1083d20bab18 None admin] RBAC: Authorizing `identity:get_project(project_id=fb99099b189242a3bc6159b227bc4e21)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:55.655586 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d131a6-d07b-4e1d-bf23-1083d20bab18 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:55.657420 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 94/193] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Aug 10 08:22:55 2022] GET /identity/v3/projects/fb99099b189242a3bc6159b227bc4e21 => generated 312 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:22:55.658115 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d131a6-d07b-4e1d-bf23-1083d20bab18 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:55.697361 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:55.700868 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d3cb00-a6a7-4a3f-8b23-5d9e3fb0d075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:55.701863 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d3cb00-a6a7-4a3f-8b23-5d9e3fb0d075 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:55.702445 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d3cb00-a6a7-4a3f-8b23-5d9e3fb0d075 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:55.702907 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d3cb00-a6a7-4a3f-8b23-5d9e3fb0d075 None admin] PATH_INFO: `/v3/users/e547c2d8da97403c8938e5a58c1003b1` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:55.704291 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d3cb00-a6a7-4a3f-8b23-5d9e3fb0d075 None admin] RBAC: Authorizing `identity:get_user(user_id=e547c2d8da97403c8938e5a58c1003b1)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:55.706088 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d3cb00-a6a7-4a3f-8b23-5d9e3fb0d075 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:55.707896 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 100/194] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Aug 10 08:22:55 2022] GET /identity/v3/users/e547c2d8da97403c8938e5a58c1003b1 => generated 283 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:22:55.708813 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d3cb00-a6a7-4a3f-8b23-5d9e3fb0d075 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:55.722226 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:55.820605 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d131a6-d07b-4e1d-bf23-1083d20bab18 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:55.857344 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d3cb00-a6a7-4a3f-8b23-5d9e3fb0d075 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:55.887635 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3513d34-6075-4a23-8908-a7628dd14e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:55.889408 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3513d34-6075-4a23-8908-a7628dd14e82 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:55.890779 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3513d34-6075-4a23-8908-a7628dd14e82 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:55.891263 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3513d34-6075-4a23-8908-a7628dd14e82 None admin] PATH_INFO: `/v3/projects/201dfacf93654f63a4732891e33d5039` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:55.894132 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3513d34-6075-4a23-8908-a7628dd14e82 None admin] RBAC: Authorizing `identity:get_project(project_id=201dfacf93654f63a4732891e33d5039)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:55.895817 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3513d34-6075-4a23-8908-a7628dd14e82 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:55.899568 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 95/195] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Aug 10 08:22:55 2022] GET /identity/v3/projects/201dfacf93654f63a4732891e33d5039 => generated 302 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:22:55.901865 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3513d34-6075-4a23-8908-a7628dd14e82 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:55.924007 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e35c75-9286-48f3-b25f-55c879c07648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:55.924690 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e35c75-9286-48f3-b25f-55c879c07648 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:55.925001 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e35c75-9286-48f3-b25f-55c879c07648 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:55.925273 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e35c75-9286-48f3-b25f-55c879c07648 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:55.926002 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e35c75-9286-48f3-b25f-55c879c07648 None admin] RBAC: Adding query filter params (role.id=44b64db484f24a46bf0b7ced46b519bf, scope.project.id=fb99099b189242a3bc6159b227bc4e21, user.id=8179b7e759db449e9295e696b2333c9c) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:22:55.926308 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e35c75-9286-48f3-b25f-55c879c07648 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:55.927739 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e35c75-9286-48f3-b25f-55c879c07648 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:55.945509 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 101/196] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Aug 10 08:22:55 2022] GET /identity/v3/role_assignments?user.id=8179b7e759db449e9295e696b2333c9c&scope.project.id=fb99099b189242a3bc6159b227bc4e21&role.id=44b64db484f24a46bf0b7ced46b519bf => generated 608 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:22:55.954668 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e35c75-9286-48f3-b25f-55c879c07648 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:56.111594 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3513d34-6075-4a23-8908-a7628dd14e82 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:56.125264 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:56.132389 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:56.181955 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e35c75-9286-48f3-b25f-55c879c07648 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:56.188033 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1631b5e-f5de-4108-844a-baa14b7d6f03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:56.188654 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1631b5e-f5de-4108-844a-baa14b7d6f03 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:56.188938 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1631b5e-f5de-4108-844a-baa14b7d6f03 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:56.189212 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1631b5e-f5de-4108-844a-baa14b7d6f03 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:56.222269 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf93533-be78-4fb5-b8c0-083db775406d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:56.225257 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf93533-be78-4fb5-b8c0-083db775406d None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:56.225257 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf93533-be78-4fb5-b8c0-083db775406d None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:56.225257 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf93533-be78-4fb5-b8c0-083db775406d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:56.225257 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf93533-be78-4fb5-b8c0-083db775406d None admin] RBAC: Adding query filter params (role.id=ec3751692d9c4ecf81b4a71c936c46be, scope.project.id=201dfacf93654f63a4732891e33d5039, user.id=e547c2d8da97403c8938e5a58c1003b1) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:22:56.225257 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf93533-be78-4fb5-b8c0-083db775406d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:56.226056 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf93533-be78-4fb5-b8c0-083db775406d None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:56.232850 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 102/197] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Aug 10 08:22:55 2022] GET /identity/v3/role_assignments?user.id=e547c2d8da97403c8938e5a58c1003b1&scope.project.id=201dfacf93654f63a4732891e33d5039&role.id=ec3751692d9c4ecf81b4a71c936c46be => generated 259 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:22:56.234157 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eecbe93-30e1-45dc-946b-2f83191095bf None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:56.234438 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eecbe93-30e1-45dc-946b-2f83191095bf None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:56.234706 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eecbe93-30e1-45dc-946b-2f83191095bf None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:56.235921 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 103/198] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:22:56 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:22:56.246862 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac88f5a0-e212-4441-b600-15c43640bf05 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:56.247214 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac88f5a0-e212-4441-b600-15c43640bf05 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:56.247404 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac88f5a0-e212-4441-b600-15c43640bf05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:56.283411 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-ac88f5a0-e212-4441-b600-15c43640bf05 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:22:56.285156 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-ac88f5a0-e212-4441-b600-15c43640bf05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:22:56.297686 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1631b5e-f5de-4108-844a-baa14b7d6f03 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:56.299667 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1631b5e-f5de-4108-844a-baa14b7d6f03 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:56.433283 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 104/199] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:22:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 08:22:56.459165 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac88f5a0-e212-4441-b600-15c43640bf05 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:56.468275 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 96/200] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Aug 10 08:22:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 566 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 08:22:56.473367 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1631b5e-f5de-4108-844a-baa14b7d6f03 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:56.476851 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:56.538028 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:56.575259 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac88f5a0-e212-4441-b600-15c43640bf05 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:56.589272 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1631b5e-f5de-4108-844a-baa14b7d6f03 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:56.616857 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278c3cc5-f00b-4668-bd2d-0bc2e8cfdfb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:56.617579 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278c3cc5-f00b-4668-bd2d-0bc2e8cfdfb6 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:56.617798 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278c3cc5-f00b-4668-bd2d-0bc2e8cfdfb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:56.618067 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278c3cc5-f00b-4668-bd2d-0bc2e8cfdfb6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:56.626259 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6b4b91-fb30-48a7-bb32-4183b0f78ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:56.627579 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6b4b91-fb30-48a7-bb32-4183b0f78ee1 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:56.628077 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6b4b91-fb30-48a7-bb32-4183b0f78ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:56.628544 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6b4b91-fb30-48a7-bb32-4183b0f78ee1 None admin] PATH_INFO: `/v3/projects/201dfacf93654f63a4732891e33d5039` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:56.629915 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6b4b91-fb30-48a7-bb32-4183b0f78ee1 None admin] RBAC: Authorizing `identity:get_project(project_id=201dfacf93654f63a4732891e33d5039)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:56.630365 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278c3cc5-f00b-4668-bd2d-0bc2e8cfdfb6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 08:22:56.631976 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6b4b91-fb30-48a7-bb32-4183b0f78ee1 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:56.634126 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 97/201] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Aug 10 08:22:56 2022] GET /identity/v3/projects/201dfacf93654f63a4732891e33d5039 => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:22:56.635130 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-278c3cc5-f00b-4668-bd2d-0bc2e8cfdfb6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 08:22:56.635130 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:22:56.635130 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:22:56.635130 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:22:56.635130 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:22:56.635130 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:22:56.635130 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:22:56.635130 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:22:56.635130 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:22:56.635130 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:22:56.635130 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:22:56.635130 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:22:56.635130 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 08:22:56.635130 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 08:22:56.635130 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 08:22:56.635130 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 08:22:56.635992 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 08:22:56.635992 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 08:22:56.635992 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 08:22:56.635992 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 08:22:56.635992 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:22:56.635992 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:22:56.635992 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:22:56.635992 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:22:56.635992 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:22:56.635992 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:22:56.635992 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:22:56.635992 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:22:56.635992 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:22:56.635992 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:22:56.635992 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:22:56.635992 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:22:56.636885 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:22:56.636885 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:22:56.636885 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:22:56.636885 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:22:56.636885 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 08:22:56.636885 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 08:22:56.636885 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 08:22:56.636885 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 08:22:56.636885 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 08:22:56.636885 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 08:22:56.636885 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 08:22:56.636885 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:22:56.636885 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 105/202] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Aug 10 08:22:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 08:22:56.640244 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6b4b91-fb30-48a7-bb32-4183b0f78ee1 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:56.641994 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278c3cc5-f00b-4668-bd2d-0bc2e8cfdfb6 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:56.729281 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:56.744824 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6b4b91-fb30-48a7-bb32-4183b0f78ee1 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:56.776992 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278c3cc5-f00b-4668-bd2d-0bc2e8cfdfb6 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:56.783232 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776b2826-f9d3-4ad3-817f-ec0fa8ad15c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:56.783917 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776b2826-f9d3-4ad3-817f-ec0fa8ad15c1 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:56.784260 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776b2826-f9d3-4ad3-817f-ec0fa8ad15c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:56.784623 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776b2826-f9d3-4ad3-817f-ec0fa8ad15c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:56.785458 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776b2826-f9d3-4ad3-817f-ec0fa8ad15c1 None admin] RBAC: Adding query filter params (role.id=ccfc8d3fdf2d458299df29001278930f, scope.project.id=201dfacf93654f63a4732891e33d5039, user.id=e547c2d8da97403c8938e5a58c1003b1) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:22:56.785832 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776b2826-f9d3-4ad3-817f-ec0fa8ad15c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:56.788648 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776b2826-f9d3-4ad3-817f-ec0fa8ad15c1 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:56.797027 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 98/203] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Aug 10 08:22:56 2022] GET /identity/v3/role_assignments?user.id=e547c2d8da97403c8938e5a58c1003b1&scope.project.id=201dfacf93654f63a4732891e33d5039&role.id=ccfc8d3fdf2d458299df29001278930f => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:22:56.813408 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:56.823621 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c91ba42-f774-4304-af0c-07ff0c9c9056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:56.824592 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c91ba42-f774-4304-af0c-07ff0c9c9056 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:56.825088 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c91ba42-f774-4304-af0c-07ff0c9c9056 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:56.826609 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c91ba42-f774-4304-af0c-07ff0c9c9056 None admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:56.826609 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c91ba42-f774-4304-af0c-07ff0c9c9056 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:22:56.826609 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c91ba42-f774-4304-af0c-07ff0c9c9056 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:56.828358 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c91ba42-f774-4304-af0c-07ff0c9c9056 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:56.839305 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 106/204] 10.222.0.57 () {56 vars in 1101 bytes} [Wed Aug 10 08:22:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:22:56.845105 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776b2826-f9d3-4ad3-817f-ec0fa8ad15c1 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:56.964563 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776b2826-f9d3-4ad3-817f-ec0fa8ad15c1 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:57.023859 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e67eb9f-933e-415b-b0dc-53e2f33a4c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:57.024442 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e67eb9f-933e-415b-b0dc-53e2f33a4c54 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:57.024716 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e67eb9f-933e-415b-b0dc-53e2f33a4c54 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:57.025015 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e67eb9f-933e-415b-b0dc-53e2f33a4c54 None admin] PATH_INFO: `/v3/users/3a70f390119d4c168dc285643fd110cc` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:57.026201 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e67eb9f-933e-415b-b0dc-53e2f33a4c54 None admin] RBAC: Authorizing `identity:get_user(user_id=3a70f390119d4c168dc285643fd110cc)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:57.027662 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e67eb9f-933e-415b-b0dc-53e2f33a4c54 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:57.029599 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 99/205] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Aug 10 08:22:56 2022] GET /identity/v3/users/3a70f390119d4c168dc285643fd110cc => generated 289 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:22:57.035141 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c91ba42-f774-4304-af0c-07ff0c9c9056 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:57.185666 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c91ba42-f774-4304-af0c-07ff0c9c9056 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:57.221316 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2913a63f-c6d4-4321-b924-9c96ac5a14c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:57.221927 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2913a63f-c6d4-4321-b924-9c96ac5a14c2 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:57.222207 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2913a63f-c6d4-4321-b924-9c96ac5a14c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:57.222478 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2913a63f-c6d4-4321-b924-9c96ac5a14c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:57.289525 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:57.297217 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2913a63f-c6d4-4321-b924-9c96ac5a14c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:57.298746 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2913a63f-c6d4-4321-b924-9c96ac5a14c2 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:57.449815 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 107/206] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Aug 10 08:22:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 08:22:57.455179 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e67eb9f-933e-415b-b0dc-53e2f33a4c54 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:57.462003 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:57.572672 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cba1452-a18f-4bc3-9b1f-a62346491f89 None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:57.573009 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cba1452-a18f-4bc3-9b1f-a62346491f89 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:57.573181 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cba1452-a18f-4bc3-9b1f-a62346491f89 None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:57.574615 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 108/207] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:22:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 08:22:57.579652 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f48637-e794-43f4-8f09-db7c66c195f4 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:57.579931 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f48637-e794-43f4-8f09-db7c66c195f4 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:57.580203 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f48637-e794-43f4-8f09-db7c66c195f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:57.595388 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e67eb9f-933e-415b-b0dc-53e2f33a4c54 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:57.617596 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-e3f48637-e794-43f4-8f09-db7c66c195f4 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:22:57.618789 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-e3f48637-e794-43f4-8f09-db7c66c195f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:22:57.638552 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46188be-3ccd-4629-8198-8af1c6d57370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:57.640157 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46188be-3ccd-4629-8198-8af1c6d57370 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:57.641318 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46188be-3ccd-4629-8198-8af1c6d57370 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:57.641895 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46188be-3ccd-4629-8198-8af1c6d57370 None admin] PATH_INFO: `/v3/projects/0f2b0269b8b24890b55cca8078954ba1` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:57.644593 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46188be-3ccd-4629-8198-8af1c6d57370 None admin] RBAC: Authorizing `identity:get_project(project_id=0f2b0269b8b24890b55cca8078954ba1)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:57.648322 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46188be-3ccd-4629-8198-8af1c6d57370 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:57.651841 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 100/208] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Aug 10 08:22:57 2022] GET /identity/v3/projects/0f2b0269b8b24890b55cca8078954ba1 => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:22:57.657627 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46188be-3ccd-4629-8198-8af1c6d57370 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:57.722275 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 109/209] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:22:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 08:22:57.744889 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f48637-e794-43f4-8f09-db7c66c195f4 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:57.773303 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:57.801314 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:57.839444 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46188be-3ccd-4629-8198-8af1c6d57370 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:57.877009 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c21f66-db75-4f8a-a5a8-29532d62dc9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:57.877618 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c21f66-db75-4f8a-a5a8-29532d62dc9f None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:57.877917 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c21f66-db75-4f8a-a5a8-29532d62dc9f None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:57.878428 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c21f66-db75-4f8a-a5a8-29532d62dc9f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:57.878985 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c21f66-db75-4f8a-a5a8-29532d62dc9f None admin] RBAC: Adding query filter params (role.id=7237967510e447498d7b37ff7d0ea7ba, scope.project.id=0f2b0269b8b24890b55cca8078954ba1, user.id=3a70f390119d4c168dc285643fd110cc) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:22:57.879293 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c21f66-db75-4f8a-a5a8-29532d62dc9f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:57.880725 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c21f66-db75-4f8a-a5a8-29532d62dc9f None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:57.888932 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f48637-e794-43f4-8f09-db7c66c195f4 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:57.896237 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 101/210] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Aug 10 08:22:57 2022] GET /identity/v3/role_assignments?user.id=3a70f390119d4c168dc285643fd110cc&scope.project.id=0f2b0269b8b24890b55cca8078954ba1&role.id=7237967510e447498d7b37ff7d0ea7ba => generated 608 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:22:57.897911 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfb8c56-925f-4f8d-8810-2c77c529e805 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:57.898197 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfb8c56-925f-4f8d-8810-2c77c529e805 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:57.898471 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfb8c56-925f-4f8d-8810-2c77c529e805 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:57.899863 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 102/211] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:22:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:22:57.912870 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4f2dd8-b8cc-454f-80c4-22abb4c21de0 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:57.912870 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4f2dd8-b8cc-454f-80c4-22abb4c21de0 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:57.912870 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4f2dd8-b8cc-454f-80c4-22abb4c21de0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:57.950937 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad666b86-5603-4480-b72c-7120c8b0c399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:57.951592 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad666b86-5603-4480-b72c-7120c8b0c399 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:57.951872 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad666b86-5603-4480-b72c-7120c8b0c399 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:57.952199 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad666b86-5603-4480-b72c-7120c8b0c399 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:57.953901 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-ea4f2dd8-b8cc-454f-80c4-22abb4c21de0 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:22:57.956570 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-ea4f2dd8-b8cc-454f-80c4-22abb4c21de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:22:57.977982 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad666b86-5603-4480-b72c-7120c8b0c399 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 08:22:57.983129 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-ad666b86-5603-4480-b72c-7120c8b0c399 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 08:22:57.983129 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:22:57.983129 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:22:57.983129 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:22:57.983129 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:22:57.983129 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:22:57.983129 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:22:57.983129 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:22:57.983129 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:22:57.983129 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:22:57.983129 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:22:57.983129 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:22:57.983129 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 08:22:57.983129 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 08:22:57.983129 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 08:22:57.983129 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 08:22:57.984082 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 08:22:57.984082 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 08:22:57.984082 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 08:22:57.984082 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 08:22:57.984082 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:22:57.984082 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:22:57.984082 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:22:57.984082 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:22:57.984082 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:22:57.984082 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:22:57.984082 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:22:57.984082 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:22:57.984082 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:22:57.984082 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:22:57.984082 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:22:57.984082 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:22:57.984881 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:22:57.984881 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:22:57.984881 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:22:57.984881 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:22:57.984881 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 08:22:57.984881 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 08:22:57.984881 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 08:22:57.984881 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 08:22:57.984881 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 08:22:57.984881 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 08:22:57.984881 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 08:22:57.984881 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:22:57.988938 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 110/212] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Aug 10 08:22:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 08:22:57.990747 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad666b86-5603-4480-b72c-7120c8b0c399 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:58.086097 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 103/213] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:22:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 08:22:58.112256 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4f2dd8-b8cc-454f-80c4-22abb4c21de0 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:58.120306 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:58.151306 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad666b86-5603-4480-b72c-7120c8b0c399 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:58.198837 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63cbdc5-942b-4e57-a3ae-8ab82fe1ac50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:58.199698 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63cbdc5-942b-4e57-a3ae-8ab82fe1ac50 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:58.199974 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63cbdc5-942b-4e57-a3ae-8ab82fe1ac50 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:58.200242 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63cbdc5-942b-4e57-a3ae-8ab82fe1ac50 None admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:58.201354 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63cbdc5-942b-4e57-a3ae-8ab82fe1ac50 None admin] RBAC: Adding query filter params (name=member) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:22:58.201797 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63cbdc5-942b-4e57-a3ae-8ab82fe1ac50 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:58.203324 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63cbdc5-942b-4e57-a3ae-8ab82fe1ac50 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:58.204601 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:58.216949 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 111/214] 10.222.0.57 () {56 vars in 1101 bytes} [Wed Aug 10 08:22:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:22:58.223110 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63cbdc5-942b-4e57-a3ae-8ab82fe1ac50 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:58.253744 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4f2dd8-b8cc-454f-80c4-22abb4c21de0 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:58.294305 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1892a97c-fdea-4e5a-8282-4c90010c3e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:58.294951 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1892a97c-fdea-4e5a-8282-4c90010c3e0f None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:58.295230 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1892a97c-fdea-4e5a-8282-4c90010c3e0f None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:58.295498 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1892a97c-fdea-4e5a-8282-4c90010c3e0f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:58.297559 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1892a97c-fdea-4e5a-8282-4c90010c3e0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:58.299515 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1892a97c-fdea-4e5a-8282-4c90010c3e0f None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:58.301893 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 104/215] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:22:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:22:58.307739 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1892a97c-fdea-4e5a-8282-4c90010c3e0f None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:58.370356 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63cbdc5-942b-4e57-a3ae-8ab82fe1ac50 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:58.420704 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657adfd0-5634-4962-900d-c04acc01fcdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:58.421655 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657adfd0-5634-4962-900d-c04acc01fcdb None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:58.422368 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657adfd0-5634-4962-900d-c04acc01fcdb None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:58.422780 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657adfd0-5634-4962-900d-c04acc01fcdb None admin] PATH_INFO: `/v3/users/06afc6db0bf24ba6a1340d84adeaa497` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:58.443019 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:58.447795 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657adfd0-5634-4962-900d-c04acc01fcdb None admin] RBAC: Authorizing `identity:get_user(user_id=06afc6db0bf24ba6a1340d84adeaa497)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:58.450104 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657adfd0-5634-4962-900d-c04acc01fcdb None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:58.453037 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 112/216] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Aug 10 08:22:58 2022] GET /identity/v3/users/06afc6db0bf24ba6a1340d84adeaa497 => generated 297 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:22:58.458885 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657adfd0-5634-4962-900d-c04acc01fcdb None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:58.464134 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1892a97c-fdea-4e5a-8282-4c90010c3e0f None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:58.510348 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8052d16f-8feb-4508-8e4c-c026cad878c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:58.511696 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8052d16f-8feb-4508-8e4c-c026cad878c7 None admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:58.512256 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8052d16f-8feb-4508-8e4c-c026cad878c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:58.512775 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8052d16f-8feb-4508-8e4c-c026cad878c7 None admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:58.513584 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8052d16f-8feb-4508-8e4c-c026cad878c7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:58.515833 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8052d16f-8feb-4508-8e4c-c026cad878c7 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:58.527352 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:58.569712 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657adfd0-5634-4962-900d-c04acc01fcdb None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:58.604948 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761e6c9d-eb2e-4043-a781-146a0b9131a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:58.605602 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761e6c9d-eb2e-4043-a781-146a0b9131a8 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:58.605886 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761e6c9d-eb2e-4043-a781-146a0b9131a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:58.606153 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761e6c9d-eb2e-4043-a781-146a0b9131a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:58.627850 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 105/217] 10.222.0.57 () {60 vars in 1132 bytes} [Wed Aug 10 08:22:58 2022] POST /identity/v3/users => generated 297 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:22:58.692424 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:58.697962 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761e6c9d-eb2e-4043-a781-146a0b9131a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:58.699954 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761e6c9d-eb2e-4043-a781-146a0b9131a8 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:58.815015 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 113/218] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Aug 10 08:22:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 10 08:22:58.821493 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8052d16f-8feb-4508-8e4c-c026cad878c7 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:58.841749 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329fdcf4-e5f0-44b1-a128-67ca9811d511 None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:58.842369 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329fdcf4-e5f0-44b1-a128-67ca9811d511 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:58.842642 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329fdcf4-e5f0-44b1-a128-67ca9811d511 None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:58.844340 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 114/219] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:22:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 08:22:58.850408 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f3f58a-b665-472a-980e-21150da7e8cc None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:58.850685 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f3f58a-b665-472a-980e-21150da7e8cc None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:58.850964 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f3f58a-b665-472a-980e-21150da7e8cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:58.875281 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-98f3f58a-b665-472a-980e-21150da7e8cc None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:22:58.876475 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-98f3f58a-b665-472a-980e-21150da7e8cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:22:58.962801 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 115/220] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:22:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 08:22:58.965568 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8052d16f-8feb-4508-8e4c-c026cad878c7 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:58.993168 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f3f58a-b665-472a-980e-21150da7e8cc None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:59.002149 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521748a9-d5da-49aa-8b06-a98336746cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:59.002789 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521748a9-d5da-49aa-8b06-a98336746cc2 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:59.003068 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521748a9-d5da-49aa-8b06-a98336746cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:59.003339 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521748a9-d5da-49aa-8b06-a98336746cc2 None admin] PATH_INFO: `/v3/projects/201dfacf93654f63a4732891e33d5039` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:59.004482 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521748a9-d5da-49aa-8b06-a98336746cc2 None admin] RBAC: Authorizing `identity:get_project(project_id=201dfacf93654f63a4732891e33d5039)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:59.005962 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521748a9-d5da-49aa-8b06-a98336746cc2 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:59.008454 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 106/221] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Aug 10 08:22:58 2022] GET /identity/v3/projects/201dfacf93654f63a4732891e33d5039 => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:22:59.014031 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521748a9-d5da-49aa-8b06-a98336746cc2 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:59.033209 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:59.091235 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:59.119465 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f3f58a-b665-472a-980e-21150da7e8cc None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:59.136641 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521748a9-d5da-49aa-8b06-a98336746cc2 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:59.157854 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3e66a8-70c6-44a4-86f6-8813dc6e2091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:59.158522 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3e66a8-70c6-44a4-86f6-8813dc6e2091 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:59.158804 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3e66a8-70c6-44a4-86f6-8813dc6e2091 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:59.159088 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3e66a8-70c6-44a4-86f6-8813dc6e2091 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:59.170155 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3e66a8-70c6-44a4-86f6-8813dc6e2091 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 08:22:59.173279 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd9f63e-2683-464c-90a3-3d01b498fc01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:59.173887 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd9f63e-2683-464c-90a3-3d01b498fc01 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:59.174161 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd9f63e-2683-464c-90a3-3d01b498fc01 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:59.174424 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd9f63e-2683-464c-90a3-3d01b498fc01 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:59.174930 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-0b3e66a8-70c6-44a4-86f6-8813dc6e2091 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 08:22:59.174930 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:22:59.174930 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:22:59.174930 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:22:59.174930 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:22:59.174930 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:22:59.174930 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:22:59.174930 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:22:59.174930 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:22:59.174930 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:22:59.174930 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:22:59.174930 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:22:59.174930 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 08:22:59.174930 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 08:22:59.174930 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 08:22:59.174930 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 08:22:59.175803 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 08:22:59.175803 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 08:22:59.175803 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 08:22:59.175803 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 08:22:59.175803 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:22:59.175803 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:22:59.175803 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:22:59.175803 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:22:59.175803 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:22:59.175803 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:22:59.175803 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:22:59.175803 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:22:59.175803 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:22:59.175803 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:22:59.175803 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:22:59.175803 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:22:59.176599 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:22:59.176599 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:22:59.176599 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:22:59.176599 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:22:59.176599 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 08:22:59.176599 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 08:22:59.176599 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 08:22:59.176599 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 08:22:59.176599 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 08:22:59.176599 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 08:22:59.176599 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 08:22:59.176599 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:22:59.177266 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd9f63e-2683-464c-90a3-3d01b498fc01 None admin] RBAC: Adding query filter params (role.id=44b64db484f24a46bf0b7ced46b519bf, scope.project.id=201dfacf93654f63a4732891e33d5039, user.id=06afc6db0bf24ba6a1340d84adeaa497) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:22:59.177266 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd9f63e-2683-464c-90a3-3d01b498fc01 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:59.177388 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 116/222] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Aug 10 08:22:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 08:22:59.177461 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd9f63e-2683-464c-90a3-3d01b498fc01 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:59.187511 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3e66a8-70c6-44a4-86f6-8813dc6e2091 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:59.187702 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 107/223] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Aug 10 08:22:59 2022] GET /identity/v3/role_assignments?user.id=06afc6db0bf24ba6a1340d84adeaa497&scope.project.id=201dfacf93654f63a4732891e33d5039&role.id=44b64db484f24a46bf0b7ced46b519bf => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:22:59.201215 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1592f3-d891-41e1-b231-6f0873e8993d None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:59.201604 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1592f3-d891-41e1-b231-6f0873e8993d None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:59.201995 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1592f3-d891-41e1-b231-6f0873e8993d None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:59.204691 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 108/224] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:22:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 08:22:59.211787 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993c79ac-4721-4594-8097-81be3db095cb None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:59.212138 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993c79ac-4721-4594-8097-81be3db095cb None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:59.212489 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993c79ac-4721-4594-8097-81be3db095cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:59.236869 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-993c79ac-4721-4594-8097-81be3db095cb None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:22:59.238262 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-993c79ac-4721-4594-8097-81be3db095cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:22:59.304348 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:59.343832 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3e66a8-70c6-44a4-86f6-8813dc6e2091 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:59.344402 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 109/225] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:22:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 08:22:59.372294 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993c79ac-4721-4594-8097-81be3db095cb None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:59.398946 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073da9f0-67eb-406d-b591-3cb7388220b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:59.398946 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073da9f0-67eb-406d-b591-3cb7388220b5 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:59.398946 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073da9f0-67eb-406d-b591-3cb7388220b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:59.398946 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073da9f0-67eb-406d-b591-3cb7388220b5 None admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:59.399355 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073da9f0-67eb-406d-b591-3cb7388220b5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:22:59.399355 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073da9f0-67eb-406d-b591-3cb7388220b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:59.399355 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073da9f0-67eb-406d-b591-3cb7388220b5 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:59.413651 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 117/226] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Aug 10 08:22:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:22:59.419158 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073da9f0-67eb-406d-b591-3cb7388220b5 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:59.478446 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:59.567128 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993c79ac-4721-4594-8097-81be3db095cb None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:59.603278 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073da9f0-67eb-406d-b591-3cb7388220b5 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:59.628578 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeffdaca-7590-4c82-9226-0893eb23882a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:59.629244 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeffdaca-7590-4c82-9226-0893eb23882a None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:59.629526 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeffdaca-7590-4c82-9226-0893eb23882a None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:59.629797 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeffdaca-7590-4c82-9226-0893eb23882a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:59.653903 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeffdaca-7590-4c82-9226-0893eb23882a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 08:22:59.661155 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-aeffdaca-7590-4c82-9226-0893eb23882a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 08:22:59.661155 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:22:59.661155 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:22:59.661155 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:22:59.661155 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:22:59.661155 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:22:59.661155 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:22:59.661155 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:22:59.661155 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:22:59.661155 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:22:59.661155 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:22:59.661155 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:22:59.661155 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 08:22:59.661155 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 08:22:59.661155 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 08:22:59.661155 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 08:22:59.662347 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 08:22:59.662347 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 08:22:59.662347 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 08:22:59.662347 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 08:22:59.662347 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:22:59.662347 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:22:59.662347 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:22:59.662347 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:22:59.662347 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:22:59.662347 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:22:59.662347 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:22:59.662347 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:22:59.662347 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:22:59.662347 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:22:59.662347 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:22:59.662347 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:22:59.663369 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:22:59.663369 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:22:59.663369 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:22:59.663369 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:22:59.663369 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 08:22:59.663369 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 08:22:59.663369 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 08:22:59.663369 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 08:22:59.663369 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 08:22:59.663369 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 08:22:59.663369 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 08:22:59.663369 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:22:59.663369 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 110/227] 10.222.0.57 () {56 vars in 1114 bytes} [Wed Aug 10 08:22:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 08:22:59.664207 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468d05e5-fe64-4868-a275-08fc0b93a6b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:59.664594 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468d05e5-fe64-4868-a275-08fc0b93a6b6 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:59.664699 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468d05e5-fe64-4868-a275-08fc0b93a6b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:59.665656 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468d05e5-fe64-4868-a275-08fc0b93a6b6 None admin] PATH_INFO: `/v3/users/e547c2d8da97403c8938e5a58c1003b1` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:59.668942 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468d05e5-fe64-4868-a275-08fc0b93a6b6 None admin] RBAC: Authorizing `identity:get_user(user_id=e547c2d8da97403c8938e5a58c1003b1)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:59.668942 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468d05e5-fe64-4868-a275-08fc0b93a6b6 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:59.669431 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 118/228] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Aug 10 08:22:59 2022] GET /identity/v3/users/e547c2d8da97403c8938e5a58c1003b1 => generated 283 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:22:59.670288 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeffdaca-7590-4c82-9226-0893eb23882a None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:59.681041 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468d05e5-fe64-4868-a275-08fc0b93a6b6 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:59.716662 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:59.858438 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeffdaca-7590-4c82-9226-0893eb23882a None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:59.863716 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468d05e5-fe64-4868-a275-08fc0b93a6b6 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:22:59.893286 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:22:59.923851 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84d3ff9-eb98-4dfd-bb57-193f8f72ec61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:59.924478 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84d3ff9-eb98-4dfd-bb57-193f8f72ec61 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:59.924759 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84d3ff9-eb98-4dfd-bb57-193f8f72ec61 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:59.925061 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84d3ff9-eb98-4dfd-bb57-193f8f72ec61 None admin] PATH_INFO: `/v3/projects/201dfacf93654f63a4732891e33d5039` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:59.926508 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84d3ff9-eb98-4dfd-bb57-193f8f72ec61 None admin] RBAC: Authorizing `identity:get_project(project_id=201dfacf93654f63a4732891e33d5039)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:59.928491 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84d3ff9-eb98-4dfd-bb57-193f8f72ec61 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:59.931708 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 119/229] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Aug 10 08:22:59 2022] GET /identity/v3/projects/201dfacf93654f63a4732891e33d5039 => generated 302 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:22:59.937979 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123e8b60-bcfc-4ea9-9141-c34599544b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:22:59.938598 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123e8b60-bcfc-4ea9-9141-c34599544b24 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:22:59.938876 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123e8b60-bcfc-4ea9-9141-c34599544b24 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:22:59.939146 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123e8b60-bcfc-4ea9-9141-c34599544b24 None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:22:59.939817 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123e8b60-bcfc-4ea9-9141-c34599544b24 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:22:59.940118 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123e8b60-bcfc-4ea9-9141-c34599544b24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:22:59.941529 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123e8b60-bcfc-4ea9-9141-c34599544b24 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:22:59.944260 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84d3ff9-eb98-4dfd-bb57-193f8f72ec61 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:22:59.958845 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 111/230] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Aug 10 08:22:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:22:59.969972 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123e8b60-bcfc-4ea9-9141-c34599544b24 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:00.042679 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:00.067608 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84d3ff9-eb98-4dfd-bb57-193f8f72ec61 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:00.139736 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e21ccaa-a972-4fd5-ba54-0aa4fda4476b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:00.140381 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e21ccaa-a972-4fd5-ba54-0aa4fda4476b None admin] REQUEST_METHOD: `PUT` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:00.140675 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e21ccaa-a972-4fd5-ba54-0aa4fda4476b None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:00.140966 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e21ccaa-a972-4fd5-ba54-0aa4fda4476b None admin] PATH_INFO: `/v3/projects/201dfacf93654f63a4732891e33d5039/users/e547c2d8da97403c8938e5a58c1003b1/roles/ec3751692d9c4ecf81b4a71c936c46be` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:00.144733 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e21ccaa-a972-4fd5-ba54-0aa4fda4476b None admin] RBAC: Authorizing `identity:create_grant(project_id=201dfacf93654f63a4732891e33d5039, user_id=e547c2d8da97403c8938e5a58c1003b1, role_id=ec3751692d9c4ecf81b4a71c936c46be)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:00.146270 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e21ccaa-a972-4fd5-ba54-0aa4fda4476b None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:00.157916 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123e8b60-bcfc-4ea9-9141-c34599544b24 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:00.194268 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb8e07d-5dc5-41c2-a2a0-d86df8fee316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:00.194928 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb8e07d-5dc5-41c2-a2a0-d86df8fee316 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:00.195276 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb8e07d-5dc5-41c2-a2a0-d86df8fee316 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:00.195546 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb8e07d-5dc5-41c2-a2a0-d86df8fee316 None admin] PATH_INFO: `/v3/users/e547c2d8da97403c8938e5a58c1003b1` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:00.196722 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb8e07d-5dc5-41c2-a2a0-d86df8fee316 None admin] RBAC: Authorizing `identity:get_user(user_id=e547c2d8da97403c8938e5a58c1003b1)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:00.198184 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb8e07d-5dc5-41c2-a2a0-d86df8fee316 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:00.199883 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 112/231] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Aug 10 08:22:59 2022] GET /identity/v3/users/e547c2d8da97403c8938e5a58c1003b1 => generated 283 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:00.205619 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb8e07d-5dc5-41c2-a2a0-d86df8fee316 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:00.225670 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 120/232] 10.222.0.57 () {58 vars in 1439 bytes} [Wed Aug 10 08:22:59 2022] PUT /identity/v3/projects/201dfacf93654f63a4732891e33d5039/users/e547c2d8da97403c8938e5a58c1003b1/roles/ec3751692d9c4ecf81b4a71c936c46be => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:23:00.255948 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:00.338758 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb8e07d-5dc5-41c2-a2a0-d86df8fee316 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:00.377170 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed85a1c-97c6-4131-acba-bfb967c2365c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:00.377821 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed85a1c-97c6-4131-acba-bfb967c2365c None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:00.378102 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed85a1c-97c6-4131-acba-bfb967c2365c None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:00.378375 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed85a1c-97c6-4131-acba-bfb967c2365c None admin] PATH_INFO: `/v3/projects/201dfacf93654f63a4732891e33d5039` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:00.379579 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed85a1c-97c6-4131-acba-bfb967c2365c None admin] RBAC: Authorizing `identity:get_project(project_id=201dfacf93654f63a4732891e33d5039)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:00.381642 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed85a1c-97c6-4131-acba-bfb967c2365c None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:00.383549 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 113/233] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Aug 10 08:23:00 2022] GET /identity/v3/projects/201dfacf93654f63a4732891e33d5039 => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:23:00.394132 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e21ccaa-a972-4fd5-ba54-0aa4fda4476b None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:00.516681 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e21ccaa-a972-4fd5-ba54-0aa4fda4476b None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:00.554200 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead02d2f-71ac-4fc7-b01e-b1aa74275320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:00.554871 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead02d2f-71ac-4fc7-b01e-b1aa74275320 None admin] REQUEST_METHOD: `PUT` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:00.555145 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead02d2f-71ac-4fc7-b01e-b1aa74275320 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:00.555413 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead02d2f-71ac-4fc7-b01e-b1aa74275320 None admin] PATH_INFO: `/v3/projects/201dfacf93654f63a4732891e33d5039/users/e547c2d8da97403c8938e5a58c1003b1/roles/ccfc8d3fdf2d458299df29001278930f` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:00.557736 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead02d2f-71ac-4fc7-b01e-b1aa74275320 None admin] RBAC: Authorizing `identity:create_grant(project_id=201dfacf93654f63a4732891e33d5039, user_id=e547c2d8da97403c8938e5a58c1003b1, role_id=ccfc8d3fdf2d458299df29001278930f)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:00.559242 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead02d2f-71ac-4fc7-b01e-b1aa74275320 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:00.663430 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 121/234] 10.222.0.57 () {58 vars in 1439 bytes} [Wed Aug 10 08:23:00 2022] PUT /identity/v3/projects/201dfacf93654f63a4732891e33d5039/users/e547c2d8da97403c8938e5a58c1003b1/roles/ccfc8d3fdf2d458299df29001278930f => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:23:01.181556 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8002e0-8767-4bae-b83e-d71ff47cab95 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:01.181556 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8002e0-8767-4bae-b83e-d71ff47cab95 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:01.181556 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8002e0-8767-4bae-b83e-d71ff47cab95 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:01.181556 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 114/235] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:23:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:23:01.186979 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f5d967-6b23-4c89-a813-db3b7bea09a1 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:01.187726 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f5d967-6b23-4c89-a813-db3b7bea09a1 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:01.188150 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f5d967-6b23-4c89-a813-db3b7bea09a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:01.207507 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:01.213799 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-99f5d967-6b23-4c89-a813-db3b7bea09a1 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:23:01.215141 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-99f5d967-6b23-4c89-a813-db3b7bea09a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:23:01.302552 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 122/236] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:23:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 08:23:01.350636 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8002e0-8767-4bae-b83e-d71ff47cab95 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:01.440170 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207403ee-9b38-42f1-bc39-be3b167e3f67 None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:01.440723 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207403ee-9b38-42f1-bc39-be3b167e3f67 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:01.441208 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207403ee-9b38-42f1-bc39-be3b167e3f67 None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:01.442749 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 123/237] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:23:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 08:23:01.449375 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4777bb07-eed5-4f89-8865-ceeb60d4be78 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:01.449895 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4777bb07-eed5-4f89-8865-ceeb60d4be78 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:01.450368 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4777bb07-eed5-4f89-8865-ceeb60d4be78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:01.474424 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-4777bb07-eed5-4f89-8865-ceeb60d4be78 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:23:01.476709 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-4777bb07-eed5-4f89-8865-ceeb60d4be78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:23:01.529807 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8002e0-8767-4bae-b83e-d71ff47cab95 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:01.549264 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:01.576880 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 124/238] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:23:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 08:23:01.582179 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ada1a3-586a-4950-93c8-d883ba85873c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:01.583948 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ada1a3-586a-4950-93c8-d883ba85873c None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:01.583948 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ada1a3-586a-4950-93c8-d883ba85873c None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:01.583948 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ada1a3-586a-4950-93c8-d883ba85873c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:01.595023 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ada1a3-586a-4950-93c8-d883ba85873c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 08:23:01.600985 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-e2ada1a3-586a-4950-93c8-d883ba85873c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 08:23:01.600985 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:01.600985 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:01.600985 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:01.600985 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:01.600985 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:01.600985 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:01.600985 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:01.600985 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:01.600985 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:01.600985 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:01.600985 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:01.600985 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 08:23:01.600985 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 08:23:01.600985 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 08:23:01.600985 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 08:23:01.601885 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 08:23:01.601885 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 08:23:01.601885 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 08:23:01.601885 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 08:23:01.601885 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:01.601885 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:01.601885 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:01.601885 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:01.601885 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:01.601885 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:01.601885 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:01.601885 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:01.601885 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:01.601885 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:01.601885 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:01.601885 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:01.602736 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:01.602736 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:01.602736 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:01.602736 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:01.602736 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 08:23:01.602736 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 08:23:01.602736 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 08:23:01.602736 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 08:23:01.602736 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 08:23:01.602736 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 08:23:01.602736 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 08:23:01.602736 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:23:01.602736 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 115/239] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Aug 10 08:23:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:01.609915 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4777bb07-eed5-4f89-8865-ceeb60d4be78 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:01.617102 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ada1a3-586a-4950-93c8-d883ba85873c None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:01.716833 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:01.770686 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4777bb07-eed5-4f89-8865-ceeb60d4be78 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:01.773782 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ada1a3-586a-4950-93c8-d883ba85873c None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:01.815161 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5711c647-aa02-41c4-8cd8-5a4f9818fa6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:01.815570 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7023d4cf-712e-49a6-a4d9-bff88f83a5d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:01.815824 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5711c647-aa02-41c4-8cd8-5a4f9818fa6d None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:01.816101 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5711c647-aa02-41c4-8cd8-5a4f9818fa6d None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:01.816211 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7023d4cf-712e-49a6-a4d9-bff88f83a5d7 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:01.816413 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5711c647-aa02-41c4-8cd8-5a4f9818fa6d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:01.816520 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7023d4cf-712e-49a6-a4d9-bff88f83a5d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:01.816684 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7023d4cf-712e-49a6-a4d9-bff88f83a5d7 None admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:01.817477 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7023d4cf-712e-49a6-a4d9-bff88f83a5d7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:01.817594 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5711c647-aa02-41c4-8cd8-5a4f9818fa6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:01.817776 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7023d4cf-712e-49a6-a4d9-bff88f83a5d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:01.818981 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5711c647-aa02-41c4-8cd8-5a4f9818fa6d None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:01.819164 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7023d4cf-712e-49a6-a4d9-bff88f83a5d7 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:01.821206 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 116/240] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:23:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:01.822347 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea032c29-1651-4226-8954-fdf34a0f0993 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:01.822626 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea032c29-1651-4226-8954-fdf34a0f0993 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:01.822897 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea032c29-1651-4226-8954-fdf34a0f0993 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:01.824096 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 117/241] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:23:01 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:23:01.830563 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea032c29-1651-4226-8954-fdf34a0f0993 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:01.837135 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 125/242] 10.222.0.57 () {56 vars in 1101 bytes} [Wed Aug 10 08:23:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:01.838458 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138ab6e4-89ad-4533-8de0-a3d4eb19bf49 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:01.838747 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138ab6e4-89ad-4533-8de0-a3d4eb19bf49 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:01.839028 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138ab6e4-89ad-4533-8de0-a3d4eb19bf49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:01.857509 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-138ab6e4-89ad-4533-8de0-a3d4eb19bf49 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:23:01.858755 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-138ab6e4-89ad-4533-8de0-a3d4eb19bf49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:23:01.885647 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:01.936998 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 126/243] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:23:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 08:23:01.936998 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138ab6e4-89ad-4533-8de0-a3d4eb19bf49 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:01.973989 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea032c29-1651-4226-8954-fdf34a0f0993 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:02.008285 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:02.013037 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72394c1b-dc5a-4aad-84cf-03b31eb58bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:02.013776 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72394c1b-dc5a-4aad-84cf-03b31eb58bb3 None admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:02.014016 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72394c1b-dc5a-4aad-84cf-03b31eb58bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:02.014340 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72394c1b-dc5a-4aad-84cf-03b31eb58bb3 None admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:02.019036 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72394c1b-dc5a-4aad-84cf-03b31eb58bb3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:02.019036 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72394c1b-dc5a-4aad-84cf-03b31eb58bb3 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:02.052296 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138ab6e4-89ad-4533-8de0-a3d4eb19bf49 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:02.087538 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401bb58c-ce0a-4986-a860-4b9b09cc5edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:02.088165 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401bb58c-ce0a-4986-a860-4b9b09cc5edb None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:02.088469 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401bb58c-ce0a-4986-a860-4b9b09cc5edb None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:02.088733 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401bb58c-ce0a-4986-a860-4b9b09cc5edb None admin] PATH_INFO: `/v3/users/b7c064680fc94d8cad8f45a64364fcdb` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:02.102542 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401bb58c-ce0a-4986-a860-4b9b09cc5edb None admin] RBAC: Authorizing `identity:get_user(user_id=b7c064680fc94d8cad8f45a64364fcdb)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:02.103992 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401bb58c-ce0a-4986-a860-4b9b09cc5edb None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:02.105887 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 127/244] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Aug 10 08:23:01 2022] GET /identity/v3/users/b7c064680fc94d8cad8f45a64364fcdb => generated 297 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:02.106437 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401bb58c-ce0a-4986-a860-4b9b09cc5edb None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:02.136868 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 118/245] 10.222.0.57 () {60 vars in 1132 bytes} [Wed Aug 10 08:23:01 2022] POST /identity/v3/users => generated 293 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:23:02.138326 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72394c1b-dc5a-4aad-84cf-03b31eb58bb3 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:02.271000 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72394c1b-dc5a-4aad-84cf-03b31eb58bb3 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:02.274865 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:02.304541 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401bb58c-ce0a-4986-a860-4b9b09cc5edb None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:02.315052 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0d4f6a-1783-40e6-8e69-cac5fad03930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:02.315830 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0d4f6a-1783-40e6-8e69-cac5fad03930 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:02.316224 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0d4f6a-1783-40e6-8e69-cac5fad03930 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:02.316623 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0d4f6a-1783-40e6-8e69-cac5fad03930 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:02.349665 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:02.368945 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c193fc-45dd-41d3-97ad-ff0061443609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:02.369599 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c193fc-45dd-41d3-97ad-ff0061443609 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:02.369879 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c193fc-45dd-41d3-97ad-ff0061443609 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:02.370156 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c193fc-45dd-41d3-97ad-ff0061443609 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:02.381920 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c193fc-45dd-41d3-97ad-ff0061443609 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 08:23:02.391968 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-b4c193fc-45dd-41d3-97ad-ff0061443609 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 08:23:02.391968 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:02.391968 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:02.391968 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:02.391968 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:02.391968 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:02.391968 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:02.391968 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:02.391968 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:02.391968 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:02.391968 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:02.391968 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:02.391968 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 08:23:02.391968 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 08:23:02.391968 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 08:23:02.391968 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 08:23:02.397769 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 08:23:02.397769 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 08:23:02.397769 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 08:23:02.397769 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 08:23:02.397769 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:02.397769 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:02.397769 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:02.397769 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:02.397769 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:02.397769 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:02.397769 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:02.397769 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:02.397769 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:02.397769 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:02.397769 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:02.397769 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:02.398711 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:02.398711 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:02.398711 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:02.398711 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:02.398711 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 08:23:02.398711 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 08:23:02.398711 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 08:23:02.398711 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 08:23:02.398711 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 08:23:02.398711 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 08:23:02.398711 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 08:23:02.398711 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:23:02.398711 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 128/246] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Aug 10 08:23:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:02.403299 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c193fc-45dd-41d3-97ad-ff0061443609 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:02.420475 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0d4f6a-1783-40e6-8e69-cac5fad03930 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:02.422147 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0d4f6a-1783-40e6-8e69-cac5fad03930 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:02.523254 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c193fc-45dd-41d3-97ad-ff0061443609 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:02.537581 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:02.551728 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 119/247] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Aug 10 08:23:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 08:23:02.553268 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0d4f6a-1783-40e6-8e69-cac5fad03930 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:02.565602 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e3ccaf-87bb-4012-ae8c-c6bbdfd56e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:02.566223 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e3ccaf-87bb-4012-ae8c-c6bbdfd56e73 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:02.566505 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e3ccaf-87bb-4012-ae8c-c6bbdfd56e73 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:02.566778 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e3ccaf-87bb-4012-ae8c-c6bbdfd56e73 None admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:02.567454 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e3ccaf-87bb-4012-ae8c-c6bbdfd56e73 None admin] RBAC: Adding query filter params (name=member) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:02.567757 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e3ccaf-87bb-4012-ae8c-c6bbdfd56e73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:02.569879 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e3ccaf-87bb-4012-ae8c-c6bbdfd56e73 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:02.584847 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 129/248] 10.222.0.57 () {56 vars in 1101 bytes} [Wed Aug 10 08:23:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:23:02.586150 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e3ccaf-87bb-4012-ae8c-c6bbdfd56e73 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:02.631746 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:02.678755 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0d4f6a-1783-40e6-8e69-cac5fad03930 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:02.710234 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e3ccaf-87bb-4012-ae8c-c6bbdfd56e73 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:02.723839 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2a4c8c-30b4-4ecd-9396-ffe889b070ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:02.725267 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2a4c8c-30b4-4ecd-9396-ffe889b070ab None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:02.725702 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2a4c8c-30b4-4ecd-9396-ffe889b070ab None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:02.726116 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2a4c8c-30b4-4ecd-9396-ffe889b070ab None admin] PATH_INFO: `/v3/projects/201dfacf93654f63a4732891e33d5039` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:02.727687 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2a4c8c-30b4-4ecd-9396-ffe889b070ab None admin] RBAC: Authorizing `identity:get_project(project_id=201dfacf93654f63a4732891e33d5039)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:02.731642 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2a4c8c-30b4-4ecd-9396-ffe889b070ab None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:02.738652 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 120/249] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Aug 10 08:23:02 2022] GET /identity/v3/projects/201dfacf93654f63a4732891e33d5039 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:23:02.738652 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1218fd16-828d-46ac-88ee-0fbe96f74e1d None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:02.738652 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1218fd16-828d-46ac-88ee-0fbe96f74e1d None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:02.738652 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1218fd16-828d-46ac-88ee-0fbe96f74e1d None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:02.739042 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 121/250] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:23:02 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:23:02.742157 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1218fd16-828d-46ac-88ee-0fbe96f74e1d None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:02.755278 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f76db95-6493-40da-bff5-9a79cc3a74ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:02.756014 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f76db95-6493-40da-bff5-9a79cc3a74ce None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:02.756306 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f76db95-6493-40da-bff5-9a79cc3a74ce None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:02.756576 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f76db95-6493-40da-bff5-9a79cc3a74ce None admin] PATH_INFO: `/v3/users/06afc6db0bf24ba6a1340d84adeaa497` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:02.758249 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f76db95-6493-40da-bff5-9a79cc3a74ce None admin] RBAC: Authorizing `identity:get_user(user_id=06afc6db0bf24ba6a1340d84adeaa497)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:02.759834 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f76db95-6493-40da-bff5-9a79cc3a74ce None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:02.761653 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 130/251] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Aug 10 08:23:02 2022] GET /identity/v3/users/06afc6db0bf24ba6a1340d84adeaa497 => generated 297 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:23:02.762916 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bd6388-0039-4d8e-b145-619324e75e63 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:02.763196 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bd6388-0039-4d8e-b145-619324e75e63 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:02.763613 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bd6388-0039-4d8e-b145-619324e75e63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:02.787102 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-d1bd6388-0039-4d8e-b145-619324e75e63 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:23:02.789384 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-d1bd6388-0039-4d8e-b145-619324e75e63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:23:02.824967 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:02.869698 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 131/252] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:23:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 08:23:02.871223 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bd6388-0039-4d8e-b145-619324e75e63 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:02.886735 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1218fd16-828d-46ac-88ee-0fbe96f74e1d None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:02.924395 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:02.931941 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdb1e8a-caa8-4a86-b2ad-e860256397c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:02.933121 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb1e8a-caa8-4a86-b2ad-e860256397c0 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:02.933379 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb1e8a-caa8-4a86-b2ad-e860256397c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:02.933662 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb1e8a-caa8-4a86-b2ad-e860256397c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:02.939158 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdb1e8a-caa8-4a86-b2ad-e860256397c0 None admin] RBAC: Adding query filter params (role.id=7237967510e447498d7b37ff7d0ea7ba, scope.project.id=201dfacf93654f63a4732891e33d5039, user.id=b7c064680fc94d8cad8f45a64364fcdb) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:02.939158 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdb1e8a-caa8-4a86-b2ad-e860256397c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:02.939158 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdb1e8a-caa8-4a86-b2ad-e860256397c0 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:02.942043 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 122/253] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Aug 10 08:23:02 2022] GET /identity/v3/role_assignments?user.id=b7c064680fc94d8cad8f45a64364fcdb&scope.project.id=201dfacf93654f63a4732891e33d5039&role.id=7237967510e447498d7b37ff7d0ea7ba => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:02.950216 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdb1e8a-caa8-4a86-b2ad-e860256397c0 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:03.018453 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bd6388-0039-4d8e-b145-619324e75e63 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:03.072757 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be77783d-9751-42e3-b854-e713f901fed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:03.073344 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be77783d-9751-42e3-b854-e713f901fed7 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:03.073619 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be77783d-9751-42e3-b854-e713f901fed7 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:03.073886 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be77783d-9751-42e3-b854-e713f901fed7 None admin] PATH_INFO: `/v3/projects/201dfacf93654f63a4732891e33d5039` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:03.076600 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be77783d-9751-42e3-b854-e713f901fed7 None admin] RBAC: Authorizing `identity:get_project(project_id=201dfacf93654f63a4732891e33d5039)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:03.078049 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be77783d-9751-42e3-b854-e713f901fed7 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:03.085566 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 132/254] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Aug 10 08:23:02 2022] GET /identity/v3/projects/201dfacf93654f63a4732891e33d5039 => generated 302 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:03.091491 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be77783d-9751-42e3-b854-e713f901fed7 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:03.184553 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdb1e8a-caa8-4a86-b2ad-e860256397c0 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:03.241165 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:03.260349 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82cb139-0482-42fd-b018-2657fb124634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:03.264940 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82cb139-0482-42fd-b018-2657fb124634 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:03.264940 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82cb139-0482-42fd-b018-2657fb124634 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:03.264940 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82cb139-0482-42fd-b018-2657fb124634 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:03.286105 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82cb139-0482-42fd-b018-2657fb124634 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 08:23:03.292778 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-e82cb139-0482-42fd-b018-2657fb124634 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 08:23:03.292778 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:03.292778 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:03.292778 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:03.292778 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:03.292778 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:03.292778 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:03.292778 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:03.292778 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:03.292778 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:03.292778 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:03.292778 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:03.292778 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 08:23:03.292778 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 08:23:03.292778 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 08:23:03.292778 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 08:23:03.293946 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 08:23:03.293946 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 08:23:03.293946 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 08:23:03.293946 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 08:23:03.293946 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:03.293946 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:03.293946 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:03.293946 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:03.293946 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:03.293946 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:03.293946 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:03.293946 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:03.293946 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:03.293946 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:03.293946 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:03.293946 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:03.294849 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:03.294849 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:03.294849 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:03.294849 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:03.294849 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 08:23:03.294849 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 08:23:03.294849 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 08:23:03.294849 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 08:23:03.294849 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 08:23:03.294849 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 08:23:03.294849 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 08:23:03.294849 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:23:03.294849 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 123/255] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Aug 10 08:23:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:03.302074 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82cb139-0482-42fd-b018-2657fb124634 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:03.303407 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be77783d-9751-42e3-b854-e713f901fed7 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:03.306424 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:03.352638 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c09f488-b6cd-411b-b928-467eac6eb1f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:03.353259 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c09f488-b6cd-411b-b928-467eac6eb1f2 None admin] REQUEST_METHOD: `PUT` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:03.353542 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c09f488-b6cd-411b-b928-467eac6eb1f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:03.353819 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c09f488-b6cd-411b-b928-467eac6eb1f2 None admin] PATH_INFO: `/v3/projects/201dfacf93654f63a4732891e33d5039/users/06afc6db0bf24ba6a1340d84adeaa497/roles/44b64db484f24a46bf0b7ced46b519bf` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:03.355860 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c09f488-b6cd-411b-b928-467eac6eb1f2 None admin] RBAC: Authorizing `identity:create_grant(project_id=201dfacf93654f63a4732891e33d5039, user_id=06afc6db0bf24ba6a1340d84adeaa497, role_id=44b64db484f24a46bf0b7ced46b519bf)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:03.357457 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c09f488-b6cd-411b-b928-467eac6eb1f2 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:03.469409 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 133/256] 10.222.0.57 () {58 vars in 1439 bytes} [Wed Aug 10 08:23:03 2022] PUT /identity/v3/projects/201dfacf93654f63a4732891e33d5039/users/06afc6db0bf24ba6a1340d84adeaa497/roles/44b64db484f24a46bf0b7ced46b519bf => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:23:03.471768 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91f43b6-e8be-4ac8-bad3-f6782d4d3dfc None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:03.472352 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91f43b6-e8be-4ac8-bad3-f6782d4d3dfc None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:03.472859 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91f43b6-e8be-4ac8-bad3-f6782d4d3dfc None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:03.474905 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 134/257] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:23:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:23:03.481271 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab0a676-df42-4c6a-8e20-34154b3c2a3d None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:03.481677 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab0a676-df42-4c6a-8e20-34154b3c2a3d None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:03.482053 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab0a676-df42-4c6a-8e20-34154b3c2a3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:03.515694 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-9ab0a676-df42-4c6a-8e20-34154b3c2a3d None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:23:03.518134 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-9ab0a676-df42-4c6a-8e20-34154b3c2a3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:23:03.520350 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82cb139-0482-42fd-b018-2657fb124634 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:03.635165 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 135/258] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:23:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 08:23:03.650362 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a120ea2b-0b05-400f-baa0-898ee2fb2cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:03.651002 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a120ea2b-0b05-400f-baa0-898ee2fb2cb5 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:03.651288 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a120ea2b-0b05-400f-baa0-898ee2fb2cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:03.651628 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a120ea2b-0b05-400f-baa0-898ee2fb2cb5 None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:03.652328 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a120ea2b-0b05-400f-baa0-898ee2fb2cb5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:03.652687 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a120ea2b-0b05-400f-baa0-898ee2fb2cb5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:03.654154 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a120ea2b-0b05-400f-baa0-898ee2fb2cb5 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:03.669767 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 124/259] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Aug 10 08:23:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 368 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:23:03.675357 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab0a676-df42-4c6a-8e20-34154b3c2a3d None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:03.687789 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a120ea2b-0b05-400f-baa0-898ee2fb2cb5 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:03.841331 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:03.844812 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab0a676-df42-4c6a-8e20-34154b3c2a3d None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:03.855651 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:03.900463 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6db0f8-d737-4638-b083-452949297332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:03.901066 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6db0f8-d737-4638-b083-452949297332 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:03.901347 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6db0f8-d737-4638-b083-452949297332 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:03.901664 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6db0f8-d737-4638-b083-452949297332 None admin] PATH_INFO: `/v3/users/e547c2d8da97403c8938e5a58c1003b1` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:03.903276 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6db0f8-d737-4638-b083-452949297332 None admin] RBAC: Authorizing `identity:get_user(user_id=e547c2d8da97403c8938e5a58c1003b1)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:03.905558 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6db0f8-d737-4638-b083-452949297332 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:03.908291 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 136/260] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Aug 10 08:23:03 2022] GET /identity/v3/users/e547c2d8da97403c8938e5a58c1003b1 => generated 283 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:03.916164 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6db0f8-d737-4638-b083-452949297332 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:03.966087 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a120ea2b-0b05-400f-baa0-898ee2fb2cb5 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:04.034441 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7478dd-1811-49bf-b459-4dbe7b8ddf34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:04.035090 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7478dd-1811-49bf-b459-4dbe7b8ddf34 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:04.035369 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7478dd-1811-49bf-b459-4dbe7b8ddf34 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:04.035639 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7478dd-1811-49bf-b459-4dbe7b8ddf34 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:04.061094 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7478dd-1811-49bf-b459-4dbe7b8ddf34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 08:23:04.072012 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-5d7478dd-1811-49bf-b459-4dbe7b8ddf34 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 08:23:04.072012 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:04.072012 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:04.072012 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:04.072012 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:04.072012 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:04.072012 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:04.072012 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:04.072012 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:04.072012 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:04.072012 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:04.072012 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:04.072012 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 08:23:04.072012 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 08:23:04.072012 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 08:23:04.072012 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 08:23:04.072966 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 08:23:04.072966 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 08:23:04.072966 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 08:23:04.072966 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 08:23:04.072966 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:04.072966 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:04.072966 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:04.072966 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:04.072966 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:04.072966 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:04.072966 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:04.072966 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:04.072966 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:04.072966 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:04.072966 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:04.072966 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:04.073782 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:04.073782 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:04.073782 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:04.073782 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:04.073782 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 08:23:04.073782 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 08:23:04.073782 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 08:23:04.073782 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 08:23:04.073782 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 08:23:04.073782 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 08:23:04.073782 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 08:23:04.073782 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:23:04.075444 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 125/261] 10.222.0.57 () {56 vars in 1114 bytes} [Wed Aug 10 08:23:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 388 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:04.081016 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7478dd-1811-49bf-b459-4dbe7b8ddf34 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:04.123460 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6db0f8-d737-4638-b083-452949297332 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:04.203616 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f842f3d-8114-48b8-bf22-0fb3c1d45f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:04.204282 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f842f3d-8114-48b8-bf22-0fb3c1d45f9d None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:04.204568 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f842f3d-8114-48b8-bf22-0fb3c1d45f9d None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:04.204845 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f842f3d-8114-48b8-bf22-0fb3c1d45f9d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:04.207974 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7478dd-1811-49bf-b459-4dbe7b8ddf34 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:04.218944 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:04.255528 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b90eb27-55c6-4ef6-b755-035b4ec71358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:04.256442 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b90eb27-55c6-4ef6-b755-035b4ec71358 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:04.256873 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b90eb27-55c6-4ef6-b755-035b4ec71358 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:04.257315 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b90eb27-55c6-4ef6-b755-035b4ec71358 None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:04.258161 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b90eb27-55c6-4ef6-b755-035b4ec71358 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:04.258614 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b90eb27-55c6-4ef6-b755-035b4ec71358 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:04.260047 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:04.260189 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b90eb27-55c6-4ef6-b755-035b4ec71358 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:04.277604 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 126/262] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Aug 10 08:23:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:04.283088 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b90eb27-55c6-4ef6-b755-035b4ec71358 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:04.304933 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f842f3d-8114-48b8-bf22-0fb3c1d45f9d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:04.306733 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f842f3d-8114-48b8-bf22-0fb3c1d45f9d None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:04.465042 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b90eb27-55c6-4ef6-b755-035b4ec71358 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:04.469982 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 137/263] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Aug 10 08:23:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 554 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 08:23:04.476307 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f842f3d-8114-48b8-bf22-0fb3c1d45f9d None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:04.531020 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72038bb-b7dc-46e9-8ba5-1924e0975911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:04.531668 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72038bb-b7dc-46e9-8ba5-1924e0975911 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:04.531954 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72038bb-b7dc-46e9-8ba5-1924e0975911 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:04.532227 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72038bb-b7dc-46e9-8ba5-1924e0975911 None admin] PATH_INFO: `/v3/users/e547c2d8da97403c8938e5a58c1003b1` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:04.538157 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72038bb-b7dc-46e9-8ba5-1924e0975911 None admin] RBAC: Authorizing `identity:get_user(user_id=e547c2d8da97403c8938e5a58c1003b1)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:04.539796 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72038bb-b7dc-46e9-8ba5-1924e0975911 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:04.541302 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 127/264] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Aug 10 08:23:04 2022] GET /identity/v3/users/e547c2d8da97403c8938e5a58c1003b1 => generated 283 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:23:04.546831 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72038bb-b7dc-46e9-8ba5-1924e0975911 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:04.590279 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:04.611007 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:04.674734 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f842f3d-8114-48b8-bf22-0fb3c1d45f9d None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:04.699230 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72038bb-b7dc-46e9-8ba5-1924e0975911 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:04.746259 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80f8bde-97b9-47ce-bfcf-fef044d922ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:04.746818 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80f8bde-97b9-47ce-bfcf-fef044d922ea None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:04.747087 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80f8bde-97b9-47ce-bfcf-fef044d922ea None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:04.747404 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80f8bde-97b9-47ce-bfcf-fef044d922ea None admin] PATH_INFO: `/v3/projects/201dfacf93654f63a4732891e33d5039` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:04.753618 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80f8bde-97b9-47ce-bfcf-fef044d922ea None admin] RBAC: Authorizing `identity:get_project(project_id=201dfacf93654f63a4732891e33d5039)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:04.755092 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80f8bde-97b9-47ce-bfcf-fef044d922ea None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:04.758355 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 138/265] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Aug 10 08:23:04 2022] GET /identity/v3/projects/201dfacf93654f63a4732891e33d5039 => generated 302 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:23:04.769893 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80f8bde-97b9-47ce-bfcf-fef044d922ea None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:04.778490 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7204ccf-73a8-4434-995a-b68f67c70943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:04.779109 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7204ccf-73a8-4434-995a-b68f67c70943 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:04.779389 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7204ccf-73a8-4434-995a-b68f67c70943 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:04.779659 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7204ccf-73a8-4434-995a-b68f67c70943 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:04.940739 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7204ccf-73a8-4434-995a-b68f67c70943 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:04.942246 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7204ccf-73a8-4434-995a-b68f67c70943 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:04.961463 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80f8bde-97b9-47ce-bfcf-fef044d922ea None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:05.018787 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:05.019284 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:05.032955 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810cdd7c-dd46-402f-9fe2-5172d1d00fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:05.032955 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810cdd7c-dd46-402f-9fe2-5172d1d00fa2 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:05.032955 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810cdd7c-dd46-402f-9fe2-5172d1d00fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:05.032955 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810cdd7c-dd46-402f-9fe2-5172d1d00fa2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:05.033363 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810cdd7c-dd46-402f-9fe2-5172d1d00fa2 None admin] RBAC: Adding query filter params (role.id=ec3751692d9c4ecf81b4a71c936c46be, scope.project.id=201dfacf93654f63a4732891e33d5039, user.id=e547c2d8da97403c8938e5a58c1003b1) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:05.033363 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810cdd7c-dd46-402f-9fe2-5172d1d00fa2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:05.034204 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810cdd7c-dd46-402f-9fe2-5172d1d00fa2 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:05.041218 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 139/266] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Aug 10 08:23:04 2022] GET /identity/v3/role_assignments?user.id=e547c2d8da97403c8938e5a58c1003b1&scope.project.id=201dfacf93654f63a4732891e33d5039&role.id=ec3751692d9c4ecf81b4a71c936c46be => generated 608 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:23:05.092459 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 128/267] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Aug 10 08:23:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 546 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 10 08:23:05.098112 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810cdd7c-dd46-402f-9fe2-5172d1d00fa2 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:05.226292 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810cdd7c-dd46-402f-9fe2-5172d1d00fa2 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:05.263763 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac657a1-9fc0-44cf-b44c-1d03ce042d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:05.264792 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac657a1-9fc0-44cf-b44c-1d03ce042d21 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:05.265304 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac657a1-9fc0-44cf-b44c-1d03ce042d21 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:05.265779 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac657a1-9fc0-44cf-b44c-1d03ce042d21 None admin] PATH_INFO: `/v3/projects/201dfacf93654f63a4732891e33d5039` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:05.267199 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac657a1-9fc0-44cf-b44c-1d03ce042d21 None admin] RBAC: Authorizing `identity:get_project(project_id=201dfacf93654f63a4732891e33d5039)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:05.269015 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac657a1-9fc0-44cf-b44c-1d03ce042d21 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:05.271981 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 140/268] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Aug 10 08:23:05 2022] GET /identity/v3/projects/201dfacf93654f63a4732891e33d5039 => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:05.282064 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7204ccf-73a8-4434-995a-b68f67c70943 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:05.423109 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7204ccf-73a8-4434-995a-b68f67c70943 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:05.464556 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef610739-45f1-4a78-b3dc-dd1dc58a67c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:05.465203 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef610739-45f1-4a78-b3dc-dd1dc58a67c7 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:05.465750 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef610739-45f1-4a78-b3dc-dd1dc58a67c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:05.465827 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef610739-45f1-4a78-b3dc-dd1dc58a67c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:05.466582 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef610739-45f1-4a78-b3dc-dd1dc58a67c7 None admin] RBAC: Adding query filter params (role.id=ccfc8d3fdf2d458299df29001278930f, scope.project.id=201dfacf93654f63a4732891e33d5039, user.id=e547c2d8da97403c8938e5a58c1003b1) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:05.466895 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef610739-45f1-4a78-b3dc-dd1dc58a67c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:05.468346 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef610739-45f1-4a78-b3dc-dd1dc58a67c7 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:05.478456 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 129/269] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Aug 10 08:23:05 2022] GET /identity/v3/role_assignments?user.id=e547c2d8da97403c8938e5a58c1003b1&scope.project.id=201dfacf93654f63a4732891e33d5039&role.id=ccfc8d3fdf2d458299df29001278930f => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:05.590230 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a17d257-7e0d-408f-bebb-b8336ab8de35 None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:05.590724 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a17d257-7e0d-408f-bebb-b8336ab8de35 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:05.590724 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a17d257-7e0d-408f-bebb-b8336ab8de35 None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:05.592312 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 141/270] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:23:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:23:05.598293 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4580c95-8382-466f-92ce-6fc2ef2a12da None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:05.598580 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4580c95-8382-466f-92ce-6fc2ef2a12da None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:05.598844 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4580c95-8382-466f-92ce-6fc2ef2a12da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:05.607920 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:05.621373 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-e4580c95-8382-466f-92ce-6fc2ef2a12da None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:23:05.622596 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-e4580c95-8382-466f-92ce-6fc2ef2a12da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:23:05.706724 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 130/271] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:23:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 08:23:05.728842 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab8d81c-bd61-4084-88c8-40e20cc18484 None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:05.729531 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab8d81c-bd61-4084-88c8-40e20cc18484 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:05.730014 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab8d81c-bd61-4084-88c8-40e20cc18484 None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:05.731952 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 142/272] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:23:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:23:05.737602 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4c73a9-a4f3-4150-b73b-9e3f1ec9cc99 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:05.737891 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4c73a9-a4f3-4150-b73b-9e3f1ec9cc99 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:05.738166 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4c73a9-a4f3-4150-b73b-9e3f1ec9cc99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:05.744810 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab8d81c-bd61-4084-88c8-40e20cc18484 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:05.762246 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-5e4c73a9-a4f3-4150-b73b-9e3f1ec9cc99 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:23:05.765361 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-5e4c73a9-a4f3-4150-b73b-9e3f1ec9cc99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:23:05.842518 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:05.867967 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 131/273] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:23:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 08:23:05.869738 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfffeebb-ae16-4f8c-b35d-824101d39af1 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:05.870277 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfffeebb-ae16-4f8c-b35d-824101d39af1 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:05.870994 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfffeebb-ae16-4f8c-b35d-824101d39af1 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:05.872798 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 132/274] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:23:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:23:05.880500 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cd5133-44c8-483c-91b5-098b71efcfdd None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:05.881088 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cd5133-44c8-483c-91b5-098b71efcfdd None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:05.882199 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cd5133-44c8-483c-91b5-098b71efcfdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:05.894751 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab8d81c-bd61-4084-88c8-40e20cc18484 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:05.906624 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-04cd5133-44c8-483c-91b5-098b71efcfdd None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:23:05.909110 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-04cd5133-44c8-483c-91b5-098b71efcfdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:23:05.932852 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ac6dd0-e18c-428d-840d-858f8f2b000b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:05.933531 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ac6dd0-e18c-428d-840d-858f8f2b000b None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:05.933814 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ac6dd0-e18c-428d-840d-858f8f2b000b None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:05.934087 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ac6dd0-e18c-428d-840d-858f8f2b000b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:05.935182 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ac6dd0-e18c-428d-840d-858f8f2b000b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:05.936654 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ac6dd0-e18c-428d-840d-858f8f2b000b None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:05.938243 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 143/275] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:23:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:23:05.939078 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ac6dd0-e18c-428d-840d-858f8f2b000b None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:05.976099 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:05.986639 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 133/276] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:23:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 08:23:05.987553 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cd5133-44c8-483c-91b5-098b71efcfdd None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:06.075004 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ac6dd0-e18c-428d-840d-858f8f2b000b None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:06.097511 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cd5133-44c8-483c-91b5-098b71efcfdd None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:06.108422 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:06.114712 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52db38d7-c016-4e0b-8670-7c1a517d8c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:06.115846 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52db38d7-c016-4e0b-8670-7c1a517d8c14 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:06.116209 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52db38d7-c016-4e0b-8670-7c1a517d8c14 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:06.121155 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52db38d7-c016-4e0b-8670-7c1a517d8c14 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:06.130512 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52db38d7-c016-4e0b-8670-7c1a517d8c14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 08:23:06.136175 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d848202e-61d2-4a1f-9645-8423ad9a6378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:06.136842 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d848202e-61d2-4a1f-9645-8423ad9a6378 None admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:06.137155 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d848202e-61d2-4a1f-9645-8423ad9a6378 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:06.137253 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-52db38d7-c016-4e0b-8670-7c1a517d8c14 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 08:23:06.137253 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:06.137253 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:06.137253 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:06.137253 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:06.137253 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:06.137253 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:06.137253 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:06.137253 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:06.137253 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:06.137253 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:06.137253 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:06.137253 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 08:23:06.137253 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 08:23:06.137253 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 08:23:06.137253 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 08:23:06.138169 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 08:23:06.138169 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 08:23:06.138169 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 08:23:06.138169 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 08:23:06.138169 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:06.138169 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:06.138169 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:06.138169 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:06.138169 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:06.138169 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:06.138169 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:06.138169 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:06.138169 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:06.138169 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:06.138169 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:06.138169 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:06.138986 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:06.138986 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:06.138986 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:06.138986 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:06.138986 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 08:23:06.138986 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 08:23:06.138986 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 08:23:06.138986 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 08:23:06.138986 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 08:23:06.138986 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 08:23:06.138986 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 08:23:06.138986 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:23:06.139603 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d848202e-61d2-4a1f-9645-8423ad9a6378 None admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:06.139603 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d848202e-61d2-4a1f-9645-8423ad9a6378 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:06.139603 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d848202e-61d2-4a1f-9645-8423ad9a6378 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:06.141052 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 144/277] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Aug 10 08:23:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:06.142044 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52db38d7-c016-4e0b-8670-7c1a517d8c14 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:06.222154 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 134/278] 10.222.0.57 () {60 vars in 1132 bytes} [Wed Aug 10 08:23:05 2022] POST /identity/v3/users => generated 293 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:23:06.223358 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d848202e-61d2-4a1f-9645-8423ad9a6378 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:06.276206 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52db38d7-c016-4e0b-8670-7c1a517d8c14 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:06.300649 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:06.330826 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6937a2-41f9-4758-a44e-a20fbfd247a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:06.331476 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6937a2-41f9-4758-a44e-a20fbfd247a8 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:06.331756 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6937a2-41f9-4758-a44e-a20fbfd247a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:06.332212 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6937a2-41f9-4758-a44e-a20fbfd247a8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:06.350078 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6937a2-41f9-4758-a44e-a20fbfd247a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 08:23:06.354815 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-8d6937a2-41f9-4758-a44e-a20fbfd247a8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 08:23:06.354815 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:06.354815 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:06.354815 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:06.354815 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:06.354815 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:06.354815 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:06.354815 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:06.354815 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:06.354815 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:06.354815 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:06.354815 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:06.354815 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 08:23:06.354815 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 08:23:06.354815 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 08:23:06.354815 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 08:23:06.355920 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 08:23:06.355920 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 08:23:06.355920 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 08:23:06.355920 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 08:23:06.355920 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:06.355920 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:06.355920 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:06.355920 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:06.355920 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:06.355920 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:06.355920 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:06.355920 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:06.355920 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:06.355920 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:06.355920 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:06.355920 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:06.356742 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:06.356742 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:06.356742 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:06.356742 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:06.356742 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 08:23:06.356742 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 08:23:06.356742 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 08:23:06.356742 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 08:23:06.356742 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 08:23:06.356742 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 08:23:06.356742 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 08:23:06.356742 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:23:06.356742 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 145/279] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Aug 10 08:23:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:06.364242 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6937a2-41f9-4758-a44e-a20fbfd247a8 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:06.398565 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d848202e-61d2-4a1f-9645-8423ad9a6378 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:06.437374 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:06.452581 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669e63b9-062a-4cc7-97fb-77649f68f24d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:06.453263 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669e63b9-062a-4cc7-97fb-77649f68f24d None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:06.453537 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669e63b9-062a-4cc7-97fb-77649f68f24d None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:06.454164 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669e63b9-062a-4cc7-97fb-77649f68f24d None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:06.454572 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669e63b9-062a-4cc7-97fb-77649f68f24d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:06.454868 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669e63b9-062a-4cc7-97fb-77649f68f24d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:06.457088 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669e63b9-062a-4cc7-97fb-77649f68f24d None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:06.466730 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 135/280] 10.222.0.57 () {56 vars in 1101 bytes} [Wed Aug 10 08:23:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:06.468444 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8750955-f08c-4eee-8aec-41097c2d202a None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:06.468444 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8750955-f08c-4eee-8aec-41097c2d202a None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:06.468706 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8750955-f08c-4eee-8aec-41097c2d202a None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:06.473489 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 136/281] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:23:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:23:06.474751 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8750955-f08c-4eee-8aec-41097c2d202a None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:06.508382 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6937a2-41f9-4758-a44e-a20fbfd247a8 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:06.557541 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdf887c-5362-498d-8e77-edc59e6b2430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:06.558164 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdf887c-5362-498d-8e77-edc59e6b2430 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:06.558448 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdf887c-5362-498d-8e77-edc59e6b2430 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:06.558722 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdf887c-5362-498d-8e77-edc59e6b2430 None admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:06.560047 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdf887c-5362-498d-8e77-edc59e6b2430 None admin] RBAC: Adding query filter params (name=member) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:06.560357 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdf887c-5362-498d-8e77-edc59e6b2430 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:06.562135 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdf887c-5362-498d-8e77-edc59e6b2430 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:06.573268 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 146/282] 10.222.0.57 () {56 vars in 1101 bytes} [Wed Aug 10 08:23:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:23:06.578264 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5895b561-197e-4988-b26a-bf96cdbef0af None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:06.578752 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5895b561-197e-4988-b26a-bf96cdbef0af None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:06.578855 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5895b561-197e-4988-b26a-bf96cdbef0af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:06.597843 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-5895b561-197e-4988-b26a-bf96cdbef0af None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:23:06.599087 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-5895b561-197e-4988-b26a-bf96cdbef0af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:23:06.615038 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8750955-f08c-4eee-8aec-41097c2d202a None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:06.631438 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:06.661135 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8da4b9-8ed3-43d1-90e5-71cc8e9c5299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:06.661920 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8da4b9-8ed3-43d1-90e5-71cc8e9c5299 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:06.662818 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8da4b9-8ed3-43d1-90e5-71cc8e9c5299 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:06.663088 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8da4b9-8ed3-43d1-90e5-71cc8e9c5299 None admin] PATH_INFO: `/v3/users/b7c064680fc94d8cad8f45a64364fcdb` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:06.664314 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8da4b9-8ed3-43d1-90e5-71cc8e9c5299 None admin] RBAC: Authorizing `identity:get_user(user_id=b7c064680fc94d8cad8f45a64364fcdb)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:06.665813 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8da4b9-8ed3-43d1-90e5-71cc8e9c5299 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:06.667818 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 137/283] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Aug 10 08:23:06 2022] GET /identity/v3/users/b7c064680fc94d8cad8f45a64364fcdb => generated 297 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:06.668639 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8da4b9-8ed3-43d1-90e5-71cc8e9c5299 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:06.684532 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 147/284] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:23:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 08:23:06.685341 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5895b561-197e-4988-b26a-bf96cdbef0af None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:06.739227 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:06.795267 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8da4b9-8ed3-43d1-90e5-71cc8e9c5299 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:06.814006 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5895b561-197e-4988-b26a-bf96cdbef0af None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:06.850290 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df18496a-7711-4191-ba11-e16d14863d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:06.851259 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df18496a-7711-4191-ba11-e16d14863d8c None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:06.851746 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df18496a-7711-4191-ba11-e16d14863d8c None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:06.851973 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3e155c-c012-4c0a-90be-81e4350cca9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:06.852388 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df18496a-7711-4191-ba11-e16d14863d8c None admin] PATH_INFO: `/v3/users/443e6b4e30d74256a01c04baa8a45e94` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:06.852754 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3e155c-c012-4c0a-90be-81e4350cca9b None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:06.853125 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3e155c-c012-4c0a-90be-81e4350cca9b None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:06.853476 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3e155c-c012-4c0a-90be-81e4350cca9b None admin] PATH_INFO: `/v3/projects/201dfacf93654f63a4732891e33d5039` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:06.854657 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3e155c-c012-4c0a-90be-81e4350cca9b None admin] RBAC: Authorizing `identity:get_project(project_id=201dfacf93654f63a4732891e33d5039)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:06.856168 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3e155c-c012-4c0a-90be-81e4350cca9b None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:06.858431 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 148/285] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Aug 10 08:23:06 2022] GET /identity/v3/projects/201dfacf93654f63a4732891e33d5039 => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:06.859296 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3e155c-c012-4c0a-90be-81e4350cca9b None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:06.872742 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df18496a-7711-4191-ba11-e16d14863d8c None admin] RBAC: Authorizing `identity:get_user(user_id=443e6b4e30d74256a01c04baa8a45e94)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:06.874876 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df18496a-7711-4191-ba11-e16d14863d8c None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:06.878053 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 138/286] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Aug 10 08:23:06 2022] GET /identity/v3/users/443e6b4e30d74256a01c04baa8a45e94 => generated 293 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:06.883043 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df18496a-7711-4191-ba11-e16d14863d8c None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:06.922200 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:06.996967 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3e155c-c012-4c0a-90be-81e4350cca9b None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:07.001025 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df18496a-7711-4191-ba11-e16d14863d8c None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:07.014740 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:07.039154 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f904201-02fa-478b-a238-59cb70ca75d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:07.039869 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f904201-02fa-478b-a238-59cb70ca75d6 None admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:07.040152 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f904201-02fa-478b-a238-59cb70ca75d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:07.040423 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f904201-02fa-478b-a238-59cb70ca75d6 None admin] PATH_INFO: `/v3/projects/201dfacf93654f63a4732891e33d5039/users/b7c064680fc94d8cad8f45a64364fcdb/roles/7237967510e447498d7b37ff7d0ea7ba` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:07.042533 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f904201-02fa-478b-a238-59cb70ca75d6 None admin] RBAC: Authorizing `identity:create_grant(project_id=201dfacf93654f63a4732891e33d5039, user_id=b7c064680fc94d8cad8f45a64364fcdb, role_id=7237967510e447498d7b37ff7d0ea7ba)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:07.044073 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f904201-02fa-478b-a238-59cb70ca75d6 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:07.048112 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f57c39-792a-49c9-ba06-4c0b3ff3b6d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:07.049280 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f57c39-792a-49c9-ba06-4c0b3ff3b6d7 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:07.049907 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f57c39-792a-49c9-ba06-4c0b3ff3b6d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:07.050415 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f57c39-792a-49c9-ba06-4c0b3ff3b6d7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:07.062170 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f57c39-792a-49c9-ba06-4c0b3ff3b6d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 08:23:07.066928 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-f2f57c39-792a-49c9-ba06-4c0b3ff3b6d7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 08:23:07.066928 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:07.066928 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:07.066928 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:07.066928 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:07.066928 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:07.066928 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:07.066928 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:07.066928 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:07.066928 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:07.066928 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:07.066928 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:07.066928 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 08:23:07.066928 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 08:23:07.066928 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 08:23:07.066928 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 08:23:07.067849 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 08:23:07.067849 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 08:23:07.067849 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 08:23:07.067849 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 08:23:07.067849 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:07.067849 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:07.067849 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:07.067849 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:07.067849 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:07.067849 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:07.067849 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:07.067849 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:07.067849 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:07.067849 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:07.067849 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:07.067849 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:07.069881 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:07.069881 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:07.069881 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:07.069881 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:07.069881 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 08:23:07.069881 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 08:23:07.069881 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 08:23:07.069881 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 08:23:07.069881 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 08:23:07.069881 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 08:23:07.069881 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 08:23:07.069881 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:23:07.069881 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 149/287] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Aug 10 08:23:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:07.073902 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f57c39-792a-49c9-ba06-4c0b3ff3b6d7 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:07.111518 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 139/288] 10.222.0.57 () {58 vars in 1439 bytes} [Wed Aug 10 08:23:06 2022] PUT /identity/v3/projects/201dfacf93654f63a4732891e33d5039/users/b7c064680fc94d8cad8f45a64364fcdb/roles/7237967510e447498d7b37ff7d0ea7ba => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:23:07.112665 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f904201-02fa-478b-a238-59cb70ca75d6 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:07.212821 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f57c39-792a-49c9-ba06-4c0b3ff3b6d7 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:07.227818 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:07.237744 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f904201-02fa-478b-a238-59cb70ca75d6 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:07.254566 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b42e4f-ec5b-4dc3-817c-d270144fe0f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:07.255297 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b42e4f-ec5b-4dc3-817c-d270144fe0f9 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:07.255738 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b42e4f-ec5b-4dc3-817c-d270144fe0f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:07.256160 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b42e4f-ec5b-4dc3-817c-d270144fe0f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:07.257091 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b42e4f-ec5b-4dc3-817c-d270144fe0f9 None admin] RBAC: Adding query filter params (role.id=44b64db484f24a46bf0b7ced46b519bf, user.id=443e6b4e30d74256a01c04baa8a45e94, scope.system=all) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:07.257498 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b42e4f-ec5b-4dc3-817c-d270144fe0f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:07.264938 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b42e4f-ec5b-4dc3-817c-d270144fe0f9 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:07.274175 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 150/289] 10.222.0.57 () {56 vars in 1308 bytes} [Wed Aug 10 08:23:07 2022] GET /identity/v3/role_assignments?user.id=443e6b4e30d74256a01c04baa8a45e94&scope.system=all&role.id=44b64db484f24a46bf0b7ced46b519bf => generated 226 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:07.306446 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed725d44-141d-4cc5-9fc2-04aae8c13e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:07.307322 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed725d44-141d-4cc5-9fc2-04aae8c13e48 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:07.307794 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed725d44-141d-4cc5-9fc2-04aae8c13e48 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:07.308135 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed725d44-141d-4cc5-9fc2-04aae8c13e48 None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:07.309009 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed725d44-141d-4cc5-9fc2-04aae8c13e48 None admin] RBAC: Adding query filter params (name=member) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:07.309428 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed725d44-141d-4cc5-9fc2-04aae8c13e48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:07.311018 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed725d44-141d-4cc5-9fc2-04aae8c13e48 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:07.325019 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 140/290] 10.222.0.57 () {56 vars in 1101 bytes} [Wed Aug 10 08:23:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:07.330905 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b42e4f-ec5b-4dc3-817c-d270144fe0f9 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:07.377618 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:07.465598 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b42e4f-ec5b-4dc3-817c-d270144fe0f9 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:07.503420 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006e0f35-e0db-4d23-9f30-bc275fea0bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:07.504057 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006e0f35-e0db-4d23-9f30-bc275fea0bbc None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:07.504337 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006e0f35-e0db-4d23-9f30-bc275fea0bbc None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:07.504607 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006e0f35-e0db-4d23-9f30-bc275fea0bbc None admin] PATH_INFO: `/v3/users/06afc6db0bf24ba6a1340d84adeaa497` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:07.505866 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006e0f35-e0db-4d23-9f30-bc275fea0bbc None admin] RBAC: Authorizing `identity:get_user(user_id=06afc6db0bf24ba6a1340d84adeaa497)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:07.507579 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006e0f35-e0db-4d23-9f30-bc275fea0bbc None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:07.509499 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 151/291] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Aug 10 08:23:07 2022] GET /identity/v3/users/06afc6db0bf24ba6a1340d84adeaa497 => generated 297 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:07.515077 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed725d44-141d-4cc5-9fc2-04aae8c13e48 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:07.653392 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed725d44-141d-4cc5-9fc2-04aae8c13e48 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:07.688189 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae733b1-fde3-4ae7-81c7-2d2170834baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:07.688802 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae733b1-fde3-4ae7-81c7-2d2170834baa None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:07.689115 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae733b1-fde3-4ae7-81c7-2d2170834baa None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:07.689441 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae733b1-fde3-4ae7-81c7-2d2170834baa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:07.735379 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:07.788961 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae733b1-fde3-4ae7-81c7-2d2170834baa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:07.790449 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae733b1-fde3-4ae7-81c7-2d2170834baa None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:07.924619 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 141/292] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Aug 10 08:23:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 08:23:07.930836 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006e0f35-e0db-4d23-9f30-bc275fea0bbc None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:08.054558 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006e0f35-e0db-4d23-9f30-bc275fea0bbc None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:08.055969 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:08.089433 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b7e87e-bb83-4de6-8231-6be167032beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:08.090065 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b7e87e-bb83-4de6-8231-6be167032beb None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:08.090348 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b7e87e-bb83-4de6-8231-6be167032beb None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:08.090615 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b7e87e-bb83-4de6-8231-6be167032beb None admin] PATH_INFO: `/v3/projects/201dfacf93654f63a4732891e33d5039` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:08.091746 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b7e87e-bb83-4de6-8231-6be167032beb None admin] RBAC: Authorizing `identity:get_project(project_id=201dfacf93654f63a4732891e33d5039)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:08.093222 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b7e87e-bb83-4de6-8231-6be167032beb None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:08.095121 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 152/293] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Aug 10 08:23:07 2022] GET /identity/v3/projects/201dfacf93654f63a4732891e33d5039 => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:08.101562 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae733b1-fde3-4ae7-81c7-2d2170834baa None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:08.179144 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:08.225931 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae733b1-fde3-4ae7-81c7-2d2170834baa None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:08.272223 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372a6ae9-b302-4ddb-8e14-0bdff2dfd0e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:08.272907 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372a6ae9-b302-4ddb-8e14-0bdff2dfd0e3 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:08.273205 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372a6ae9-b302-4ddb-8e14-0bdff2dfd0e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:08.273526 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372a6ae9-b302-4ddb-8e14-0bdff2dfd0e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:08.274257 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372a6ae9-b302-4ddb-8e14-0bdff2dfd0e3 None admin] RBAC: Adding query filter params (role.id=44b64db484f24a46bf0b7ced46b519bf, scope.project.id=201dfacf93654f63a4732891e33d5039, user.id=06afc6db0bf24ba6a1340d84adeaa497) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:08.274565 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372a6ae9-b302-4ddb-8e14-0bdff2dfd0e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:08.276003 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372a6ae9-b302-4ddb-8e14-0bdff2dfd0e3 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:08.285533 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 142/294] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Aug 10 08:23:08 2022] GET /identity/v3/role_assignments?user.id=06afc6db0bf24ba6a1340d84adeaa497&scope.project.id=201dfacf93654f63a4732891e33d5039&role.id=44b64db484f24a46bf0b7ced46b519bf => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:08.702113 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1884fcca-78af-4c20-b526-efb421c89a68 None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:08.702490 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1884fcca-78af-4c20-b526-efb421c89a68 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:08.702669 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1884fcca-78af-4c20-b526-efb421c89a68 None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:08.704226 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 153/295] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:23:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:23:08.710202 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53216c70-6908-4f02-a6af-53ce0bf8cd4e None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:08.710660 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53216c70-6908-4f02-a6af-53ce0bf8cd4e None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:08.710836 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53216c70-6908-4f02-a6af-53ce0bf8cd4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:08.730201 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-53216c70-6908-4f02-a6af-53ce0bf8cd4e None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:23:08.731440 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-53216c70-6908-4f02-a6af-53ce0bf8cd4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:23:08.807727 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 143/296] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:23:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 08:23:08.832990 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1884fcca-78af-4c20-b526-efb421c89a68 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:08.893926 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8552e6f3-4e11-479f-b360-0093ea03c1d3 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:08.894229 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8552e6f3-4e11-479f-b360-0093ea03c1d3 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:08.894473 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8552e6f3-4e11-479f-b360-0093ea03c1d3 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:08.895813 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 144/297] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:23:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 08:23:08.903063 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed5ee2d-6568-416d-8507-fe5e651ee18c None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:08.903332 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed5ee2d-6568-416d-8507-fe5e651ee18c None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:08.903604 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed5ee2d-6568-416d-8507-fe5e651ee18c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:08.924082 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-3ed5ee2d-6568-416d-8507-fe5e651ee18c None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:23:08.925824 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-3ed5ee2d-6568-416d-8507-fe5e651ee18c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:23:08.965820 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1884fcca-78af-4c20-b526-efb421c89a68 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:09.001373 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:09.008109 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a829c6df-0681-47fe-8305-3c0defb0303f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:09.009127 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a829c6df-0681-47fe-8305-3c0defb0303f None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:09.009580 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a829c6df-0681-47fe-8305-3c0defb0303f None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:09.009884 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 145/298] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:23:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 08:23:09.015201 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a829c6df-0681-47fe-8305-3c0defb0303f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:09.023307 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a829c6df-0681-47fe-8305-3c0defb0303f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 08:23:09.028723 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-a829c6df-0681-47fe-8305-3c0defb0303f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 08:23:09.028723 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:09.028723 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:09.028723 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:09.028723 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:09.028723 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:09.028723 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:09.028723 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:09.028723 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:09.028723 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:09.028723 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:09.028723 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:09.028723 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 08:23:09.028723 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 08:23:09.028723 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 08:23:09.028723 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 08:23:09.029659 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 08:23:09.029659 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 08:23:09.029659 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 08:23:09.029659 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 08:23:09.029659 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:09.029659 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:09.029659 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:09.029659 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:09.029659 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:09.029659 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:09.029659 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:09.029659 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:09.029659 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:09.029659 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:09.029659 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:09.029659 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:09.030464 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:09.030464 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:09.030464 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:09.030464 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:09.030464 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 08:23:09.030464 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 08:23:09.030464 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 08:23:09.030464 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 08:23:09.030464 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 08:23:09.030464 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 08:23:09.030464 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 08:23:09.030464 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:23:09.033172 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 154/299] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Aug 10 08:23:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:09.033859 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed5ee2d-6568-416d-8507-fe5e651ee18c None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:09.040265 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a829c6df-0681-47fe-8305-3c0defb0303f None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:09.085924 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:09.155918 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed5ee2d-6568-416d-8507-fe5e651ee18c None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:09.176687 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a829c6df-0681-47fe-8305-3c0defb0303f None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:09.195271 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d1fb1a-6e03-4040-92bf-d926c15b23b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:09.196029 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d1fb1a-6e03-4040-92bf-d926c15b23b3 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:09.196257 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d1fb1a-6e03-4040-92bf-d926c15b23b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:09.196533 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d1fb1a-6e03-4040-92bf-d926c15b23b3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:09.197732 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d1fb1a-6e03-4040-92bf-d926c15b23b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:09.199213 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d1fb1a-6e03-4040-92bf-d926c15b23b3 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:09.205080 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 146/300] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:23:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:09.207051 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d1fb1a-6e03-4040-92bf-d926c15b23b3 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:09.214742 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee3266f-479a-4f9d-b8dd-24d1d07e276a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:09.215619 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee3266f-479a-4f9d-b8dd-24d1d07e276a None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:09.216191 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee3266f-479a-4f9d-b8dd-24d1d07e276a None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:09.216659 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee3266f-479a-4f9d-b8dd-24d1d07e276a None admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:09.217560 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee3266f-479a-4f9d-b8dd-24d1d07e276a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:09.218063 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee3266f-479a-4f9d-b8dd-24d1d07e276a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:09.219619 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee3266f-479a-4f9d-b8dd-24d1d07e276a None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:09.233854 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 155/301] 10.222.0.57 () {56 vars in 1101 bytes} [Wed Aug 10 08:23:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:23:09.235036 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee3266f-479a-4f9d-b8dd-24d1d07e276a None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:09.261959 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:09.315265 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d1fb1a-6e03-4040-92bf-d926c15b23b3 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:09.329416 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:09.338689 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee3266f-479a-4f9d-b8dd-24d1d07e276a None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:09.354978 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c12ce83-e63a-4f3b-9150-7583df14704a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:09.355828 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c12ce83-e63a-4f3b-9150-7583df14704a None admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:09.356253 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c12ce83-e63a-4f3b-9150-7583df14704a None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:09.356667 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c12ce83-e63a-4f3b-9150-7583df14704a None admin] PATH_INFO: `/v3/groups` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:09.358644 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c12ce83-e63a-4f3b-9150-7583df14704a None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:09.362754 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c12ce83-e63a-4f3b-9150-7583df14704a None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:09.376241 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3428b6-1700-4ce7-aacf-47d5bf3c22e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:09.376999 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3428b6-1700-4ce7-aacf-47d5bf3c22e9 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:09.377430 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3428b6-1700-4ce7-aacf-47d5bf3c22e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:09.377769 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3428b6-1700-4ce7-aacf-47d5bf3c22e9 None admin] PATH_INFO: `/v3/users/a49ae0c00a214b9c8d12294a561d6ce2` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:09.457297 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 147/302] 10.222.0.57 () {60 vars in 1134 bytes} [Wed Aug 10 08:23:09 2022] POST /identity/v3/groups => generated 229 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:23:09.529931 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3428b6-1700-4ce7-aacf-47d5bf3c22e9 None admin] RBAC: Authorizing `identity:get_user(user_id=a49ae0c00a214b9c8d12294a561d6ce2)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:09.531775 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3428b6-1700-4ce7-aacf-47d5bf3c22e9 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:09.534042 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 156/303] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Aug 10 08:23:09 2022] GET /identity/v3/users/a49ae0c00a214b9c8d12294a561d6ce2 => generated 293 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:09.541302 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c12ce83-e63a-4f3b-9150-7583df14704a None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:09.555617 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4439a21-c874-4d5e-8f47-6fe730c11016 None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:09.555617 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4439a21-c874-4d5e-8f47-6fe730c11016 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:09.555876 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4439a21-c874-4d5e-8f47-6fe730c11016 None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:09.557160 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 157/304] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:23:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:23:09.562603 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf67fab9-6009-4f5b-9601-56d36df98f46 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:09.562911 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf67fab9-6009-4f5b-9601-56d36df98f46 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:09.563155 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf67fab9-6009-4f5b-9601-56d36df98f46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:09.581721 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-bf67fab9-6009-4f5b-9601-56d36df98f46 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:23:09.583068 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-bf67fab9-6009-4f5b-9601-56d36df98f46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:23:09.663331 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 158/305] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:23:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 08:23:09.678714 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c12ce83-e63a-4f3b-9150-7583df14704a None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:09.686115 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf67fab9-6009-4f5b-9601-56d36df98f46 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:09.690988 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:09.724624 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d176df26-7660-4d58-8825-292b2d836e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:09.726811 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d176df26-7660-4d58-8825-292b2d836e1b None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:09.727252 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d176df26-7660-4d58-8825-292b2d836e1b None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:09.727525 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d176df26-7660-4d58-8825-292b2d836e1b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:09.728270 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d176df26-7660-4d58-8825-292b2d836e1b None admin] RBAC: Adding query filter params (role.id=7237967510e447498d7b37ff7d0ea7ba, user.id=a49ae0c00a214b9c8d12294a561d6ce2, scope.system=all) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:09.728576 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d176df26-7660-4d58-8825-292b2d836e1b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:09.730072 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d176df26-7660-4d58-8825-292b2d836e1b None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:09.737302 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 148/306] 10.222.0.57 () {56 vars in 1308 bytes} [Wed Aug 10 08:23:09 2022] GET /identity/v3/role_assignments?user.id=a49ae0c00a214b9c8d12294a561d6ce2&scope.system=all&role.id=7237967510e447498d7b37ff7d0ea7ba => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:09.781541 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:09.803743 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf67fab9-6009-4f5b-9601-56d36df98f46 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:09.843188 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c55d7c0-81e9-4e5d-9f7a-25ac82f9581a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:09.843833 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c55d7c0-81e9-4e5d-9f7a-25ac82f9581a None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:09.844112 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c55d7c0-81e9-4e5d-9f7a-25ac82f9581a None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:09.844451 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c55d7c0-81e9-4e5d-9f7a-25ac82f9581a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:09.857016 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c55d7c0-81e9-4e5d-9f7a-25ac82f9581a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 08:23:09.861993 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-8c55d7c0-81e9-4e5d-9f7a-25ac82f9581a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 08:23:09.861993 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:09.861993 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:09.861993 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:09.861993 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:09.861993 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:09.861993 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:09.861993 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:09.861993 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:09.861993 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:09.861993 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:09.861993 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:09.861993 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 08:23:09.861993 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 08:23:09.861993 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 08:23:09.861993 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 08:23:09.862930 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 08:23:09.862930 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 08:23:09.862930 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 08:23:09.862930 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 08:23:09.862930 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:09.862930 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:09.862930 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:09.862930 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:09.862930 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:09.862930 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:09.862930 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:09.862930 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:09.862930 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:09.862930 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:09.862930 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:09.862930 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:09.863734 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:09.863734 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:09.863734 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:09.863734 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:09.863734 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 08:23:09.863734 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 08:23:09.863734 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 08:23:09.863734 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 08:23:09.863734 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 08:23:09.863734 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 08:23:09.863734 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 08:23:09.863734 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:23:09.863734 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 159/307] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Aug 10 08:23:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 08:23:09.870284 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d176df26-7660-4d58-8825-292b2d836e1b None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:09.925460 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc6ce97-1606-49c5-ac08-44ab13dc227f None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:09.926038 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc6ce97-1606-49c5-ac08-44ab13dc227f None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:09.926503 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc6ce97-1606-49c5-ac08-44ab13dc227f None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:09.928281 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 160/308] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:23:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:23:09.934435 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7854373-2993-45f6-ad37-f0732a3f7dc8 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:09.934914 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7854373-2993-45f6-ad37-f0732a3f7dc8 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:09.935194 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7854373-2993-45f6-ad37-f0732a3f7dc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:09.961880 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-b7854373-2993-45f6-ad37-f0732a3f7dc8 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:23:09.964239 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-b7854373-2993-45f6-ad37-f0732a3f7dc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:23:09.989264 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d176df26-7660-4d58-8825-292b2d836e1b None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:10.032964 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2785cb3-7350-45fa-ad06-0aa87a9fc923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:10.034319 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2785cb3-7350-45fa-ad06-0aa87a9fc923 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:10.034780 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2785cb3-7350-45fa-ad06-0aa87a9fc923 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:10.035234 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2785cb3-7350-45fa-ad06-0aa87a9fc923 None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:10.036375 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2785cb3-7350-45fa-ad06-0aa87a9fc923 None admin] RBAC: Adding query filter params (name=member) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:10.037469 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2785cb3-7350-45fa-ad06-0aa87a9fc923 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:10.039403 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2785cb3-7350-45fa-ad06-0aa87a9fc923 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:10.051308 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 149/309] 10.222.0.57 () {56 vars in 1101 bytes} [Wed Aug 10 08:23:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:10.054907 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 161/310] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:23:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 08:23:10.057991 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2785cb3-7350-45fa-ad06-0aa87a9fc923 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:10.077691 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7854373-2993-45f6-ad37-f0732a3f7dc8 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:10.110668 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:10.163399 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:10.199066 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2785cb3-7350-45fa-ad06-0aa87a9fc923 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:10.208259 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7854373-2993-45f6-ad37-f0732a3f7dc8 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:10.241412 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e92335-9f3e-4ace-8f71-e9829d789b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:10.243075 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e92335-9f3e-4ace-8f71-e9829d789b43 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:10.243676 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e92335-9f3e-4ace-8f71-e9829d789b43 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:10.244221 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e92335-9f3e-4ace-8f71-e9829d789b43 None admin] PATH_INFO: `/v3/users/443e6b4e30d74256a01c04baa8a45e94` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:10.245642 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e92335-9f3e-4ace-8f71-e9829d789b43 None admin] RBAC: Authorizing `identity:get_user(user_id=443e6b4e30d74256a01c04baa8a45e94)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:10.247596 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e92335-9f3e-4ace-8f71-e9829d789b43 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:10.249943 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 150/311] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Aug 10 08:23:10 2022] GET /identity/v3/users/443e6b4e30d74256a01c04baa8a45e94 => generated 293 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:10.253255 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b82c229-e37f-4320-a7df-6e714879919b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:10.253901 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b82c229-e37f-4320-a7df-6e714879919b None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:10.254180 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b82c229-e37f-4320-a7df-6e714879919b None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:10.254451 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b82c229-e37f-4320-a7df-6e714879919b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:10.257405 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e92335-9f3e-4ace-8f71-e9829d789b43 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:10.266902 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b82c229-e37f-4320-a7df-6e714879919b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 08:23:10.272604 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-7b82c229-e37f-4320-a7df-6e714879919b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 08:23:10.272604 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:10.272604 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:10.272604 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:10.272604 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:10.272604 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:10.272604 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:10.272604 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:10.272604 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:10.272604 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:10.272604 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:10.272604 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:10.272604 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 08:23:10.272604 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 08:23:10.272604 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 08:23:10.272604 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 08:23:10.273546 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 08:23:10.273546 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 08:23:10.273546 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 08:23:10.273546 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 08:23:10.273546 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:10.273546 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:10.273546 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:10.273546 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:10.273546 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:10.273546 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:10.273546 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:10.273546 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:10.273546 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:10.273546 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:10.273546 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:10.273546 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:10.274359 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:10.274359 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:10.274359 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:10.274359 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:10.274359 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 08:23:10.274359 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 08:23:10.274359 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 08:23:10.274359 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 08:23:10.274359 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 08:23:10.274359 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 08:23:10.274359 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 08:23:10.274359 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:23:10.276902 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 162/312] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Aug 10 08:23:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 08:23:10.283584 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b82c229-e37f-4320-a7df-6e714879919b None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:10.386724 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e92335-9f3e-4ace-8f71-e9829d789b43 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:10.390183 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:10.421785 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b82c229-e37f-4320-a7df-6e714879919b None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:10.423279 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b81cd2-9cd7-428c-944f-75357766ce0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:10.423984 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b81cd2-9cd7-428c-944f-75357766ce0b None admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:10.424266 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b81cd2-9cd7-428c-944f-75357766ce0b None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:10.426377 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b81cd2-9cd7-428c-944f-75357766ce0b None admin] PATH_INFO: `/v3/system/users/443e6b4e30d74256a01c04baa8a45e94/roles/44b64db484f24a46bf0b7ced46b519bf` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:10.426377 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b81cd2-9cd7-428c-944f-75357766ce0b None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=443e6b4e30d74256a01c04baa8a45e94, role_id=44b64db484f24a46bf0b7ced46b519bf)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:10.428545 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b81cd2-9cd7-428c-944f-75357766ce0b None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:10.453091 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:10.459555 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baf8651-7070-4d78-9615-6ba3aa707062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:10.460170 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baf8651-7070-4d78-9615-6ba3aa707062 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:10.460443 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baf8651-7070-4d78-9615-6ba3aa707062 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:10.460724 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baf8651-7070-4d78-9615-6ba3aa707062 None admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:10.461400 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baf8651-7070-4d78-9615-6ba3aa707062 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:10.461698 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baf8651-7070-4d78-9615-6ba3aa707062 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:10.463052 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baf8651-7070-4d78-9615-6ba3aa707062 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:10.471804 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 163/313] 10.222.0.57 () {56 vars in 1101 bytes} [Wed Aug 10 08:23:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:10.477623 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baf8651-7070-4d78-9615-6ba3aa707062 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:10.530807 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 151/314] 10.222.0.57 () {58 vars in 1334 bytes} [Wed Aug 10 08:23:10 2022] PUT /identity/v3/system/users/443e6b4e30d74256a01c04baa8a45e94/roles/44b64db484f24a46bf0b7ced46b519bf => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 08:23:10.588977 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baf8651-7070-4d78-9615-6ba3aa707062 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:10.627476 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f670a778-12a5-486a-b808-319a4133bc53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:10.628121 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f670a778-12a5-486a-b808-319a4133bc53 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:10.628395 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f670a778-12a5-486a-b808-319a4133bc53 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:10.628659 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f670a778-12a5-486a-b808-319a4133bc53 None admin] PATH_INFO: `/v3/users/b7c064680fc94d8cad8f45a64364fcdb` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:10.629965 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f670a778-12a5-486a-b808-319a4133bc53 None admin] RBAC: Authorizing `identity:get_user(user_id=b7c064680fc94d8cad8f45a64364fcdb)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:10.631495 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f670a778-12a5-486a-b808-319a4133bc53 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:10.633522 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 164/315] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Aug 10 08:23:10 2022] GET /identity/v3/users/b7c064680fc94d8cad8f45a64364fcdb => generated 297 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:23:10.639201 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b81cd2-9cd7-428c-944f-75357766ce0b None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:10.748717 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b81cd2-9cd7-428c-944f-75357766ce0b None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:10.787358 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46151be-40c0-4afd-a27b-f0d2cb27ac82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:10.787982 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46151be-40c0-4afd-a27b-f0d2cb27ac82 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:10.788264 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46151be-40c0-4afd-a27b-f0d2cb27ac82 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:10.788536 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46151be-40c0-4afd-a27b-f0d2cb27ac82 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:10.830156 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:10.864945 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46151be-40c0-4afd-a27b-f0d2cb27ac82 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:10.866828 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46151be-40c0-4afd-a27b-f0d2cb27ac82 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:10.969706 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 152/316] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Aug 10 08:23:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 08:23:10.975880 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f670a778-12a5-486a-b808-319a4133bc53 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:11.045352 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:11.079571 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f670a778-12a5-486a-b808-319a4133bc53 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:11.114083 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015e98c9-f465-42bd-bff3-2ca86ee814f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:11.115635 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015e98c9-f465-42bd-bff3-2ca86ee814f7 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:11.116158 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015e98c9-f465-42bd-bff3-2ca86ee814f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:11.116617 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015e98c9-f465-42bd-bff3-2ca86ee814f7 None admin] PATH_INFO: `/v3/projects/201dfacf93654f63a4732891e33d5039` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:11.117911 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015e98c9-f465-42bd-bff3-2ca86ee814f7 None admin] RBAC: Authorizing `identity:get_project(project_id=201dfacf93654f63a4732891e33d5039)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:11.119492 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015e98c9-f465-42bd-bff3-2ca86ee814f7 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:11.121576 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 165/317] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Aug 10 08:23:10 2022] GET /identity/v3/projects/201dfacf93654f63a4732891e33d5039 => generated 302 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:11.127647 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46151be-40c0-4afd-a27b-f0d2cb27ac82 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:11.196631 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:11.230585 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46151be-40c0-4afd-a27b-f0d2cb27ac82 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:11.266632 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697169f5-3be5-4aa1-b72e-0f20045be55c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:11.267239 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697169f5-3be5-4aa1-b72e-0f20045be55c None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:11.267516 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697169f5-3be5-4aa1-b72e-0f20045be55c None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:11.267791 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697169f5-3be5-4aa1-b72e-0f20045be55c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:11.268507 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697169f5-3be5-4aa1-b72e-0f20045be55c None admin] RBAC: Adding query filter params (role.id=7237967510e447498d7b37ff7d0ea7ba, scope.project.id=201dfacf93654f63a4732891e33d5039, user.id=b7c064680fc94d8cad8f45a64364fcdb) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:11.268812 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697169f5-3be5-4aa1-b72e-0f20045be55c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:11.270316 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697169f5-3be5-4aa1-b72e-0f20045be55c None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:11.276733 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 153/318] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Aug 10 08:23:11 2022] GET /identity/v3/role_assignments?user.id=b7c064680fc94d8cad8f45a64364fcdb&scope.project.id=201dfacf93654f63a4732891e33d5039&role.id=7237967510e447498d7b37ff7d0ea7ba => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:11.708416 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a6af47-f4be-4e4b-94c8-04aa6c2a231e None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:11.708909 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a6af47-f4be-4e4b-94c8-04aa6c2a231e None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:11.709055 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a6af47-f4be-4e4b-94c8-04aa6c2a231e None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:11.710552 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 166/319] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:23:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:23:11.715479 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d101d100-e45d-405a-81ee-b430c394c467 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:11.715772 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d101d100-e45d-405a-81ee-b430c394c467 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:11.716047 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d101d100-e45d-405a-81ee-b430c394c467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:11.734569 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-d101d100-e45d-405a-81ee-b430c394c467 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:23:11.735755 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-d101d100-e45d-405a-81ee-b430c394c467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:23:11.805433 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 154/320] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:23:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 08:23:11.827809 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a6af47-f4be-4e4b-94c8-04aa6c2a231e None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:11.910717 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aa53ac-b609-48a5-a393-e4b6d9109f39 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:11.910999 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aa53ac-b609-48a5-a393-e4b6d9109f39 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:11.911271 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aa53ac-b609-48a5-a393-e4b6d9109f39 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:11.913505 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 155/321] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:23:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 08:23:11.918133 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c1f4fe-930e-4cdb-a212-de163dac1e77 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:11.918413 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c1f4fe-930e-4cdb-a212-de163dac1e77 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:11.918681 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c1f4fe-930e-4cdb-a212-de163dac1e77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:11.933777 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a6af47-f4be-4e4b-94c8-04aa6c2a231e None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:11.936588 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-a5c1f4fe-930e-4cdb-a212-de163dac1e77 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:23:11.937466 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-a5c1f4fe-930e-4cdb-a212-de163dac1e77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:23:11.963029 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:11.971021 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b5e0ca-8007-4e1f-8af3-29332f689e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:11.971639 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b5e0ca-8007-4e1f-8af3-29332f689e14 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:11.971913 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b5e0ca-8007-4e1f-8af3-29332f689e14 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:11.972174 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b5e0ca-8007-4e1f-8af3-29332f689e14 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:11.973293 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b5e0ca-8007-4e1f-8af3-29332f689e14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:11.975501 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b5e0ca-8007-4e1f-8af3-29332f689e14 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:11.981368 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 167/322] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:23:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:11.983074 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b5e0ca-8007-4e1f-8af3-29332f689e14 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:11.999959 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:12.010618 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 156/323] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:23:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 08:23:12.032309 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c1f4fe-930e-4cdb-a212-de163dac1e77 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:12.089442 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b5e0ca-8007-4e1f-8af3-29332f689e14 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:12.126569 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9675b2-a90b-4787-abe3-0dec53909cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:12.127173 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9675b2-a90b-4787-abe3-0dec53909cde None admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:12.127447 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9675b2-a90b-4787-abe3-0dec53909cde None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:12.127714 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9675b2-a90b-4787-abe3-0dec53909cde None admin] PATH_INFO: `/v3/groups` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:12.128333 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9675b2-a90b-4787-abe3-0dec53909cde None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:12.129803 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9675b2-a90b-4787-abe3-0dec53909cde None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:12.145581 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c1f4fe-930e-4cdb-a212-de163dac1e77 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:12.180228 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c22f527-5d2d-4c76-8337-bdc68b769218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:12.180858 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c22f527-5d2d-4c76-8337-bdc68b769218 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:12.181185 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c22f527-5d2d-4c76-8337-bdc68b769218 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:12.181454 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c22f527-5d2d-4c76-8337-bdc68b769218 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:12.191282 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c22f527-5d2d-4c76-8337-bdc68b769218 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 08:23:12.195958 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-8c22f527-5d2d-4c76-8337-bdc68b769218 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 08:23:12.195958 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:12.195958 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:12.195958 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:12.195958 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:12.195958 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:12.195958 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:12.195958 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:12.195958 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:12.195958 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:12.195958 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:12.195958 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:12.195958 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 08:23:12.195958 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 08:23:12.195958 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 08:23:12.195958 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 08:23:12.198922 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 08:23:12.198922 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 08:23:12.198922 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 08:23:12.198922 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 08:23:12.198922 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:12.198922 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:12.198922 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:12.198922 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:12.198922 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:12.198922 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:12.198922 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:12.198922 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:12.198922 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:12.198922 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:12.198922 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:12.198922 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:12.199948 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:12.199948 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:12.199948 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:12.199948 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:12.199948 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 08:23:12.199948 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 08:23:12.199948 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 08:23:12.199948 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 08:23:12.199948 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 08:23:12.199948 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 08:23:12.199948 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 08:23:12.199948 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:23:12.199948 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 157/324] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Aug 10 08:23:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 08:23:12.202592 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c22f527-5d2d-4c76-8337-bdc68b769218 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:12.220022 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 168/325] 10.222.0.57 () {60 vars in 1134 bytes} [Wed Aug 10 08:23:11 2022] POST /identity/v3/groups => generated 226 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 08:23:12.274025 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:12.327638 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c22f527-5d2d-4c76-8337-bdc68b769218 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:12.364441 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aa12f4-882d-45a5-9554-990a9b519548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:12.365705 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aa12f4-882d-45a5-9554-990a9b519548 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:12.365981 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aa12f4-882d-45a5-9554-990a9b519548 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:12.366248 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aa12f4-882d-45a5-9554-990a9b519548 None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:12.366920 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aa12f4-882d-45a5-9554-990a9b519548 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:12.367220 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aa12f4-882d-45a5-9554-990a9b519548 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:12.369468 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aa12f4-882d-45a5-9554-990a9b519548 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:12.380030 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 158/326] 10.222.0.57 () {56 vars in 1101 bytes} [Wed Aug 10 08:23:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:23:12.385425 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9675b2-a90b-4787-abe3-0dec53909cde None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:12.448778 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:12.494739 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9675b2-a90b-4787-abe3-0dec53909cde None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:12.529508 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e859550-1578-4647-b340-e2de7d09438e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:12.530146 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e859550-1578-4647-b340-e2de7d09438e None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:12.530426 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e859550-1578-4647-b340-e2de7d09438e None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:12.530696 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e859550-1578-4647-b340-e2de7d09438e None admin] PATH_INFO: `/v3/users/a49ae0c00a214b9c8d12294a561d6ce2` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:12.532798 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e859550-1578-4647-b340-e2de7d09438e None admin] RBAC: Authorizing `identity:get_user(user_id=a49ae0c00a214b9c8d12294a561d6ce2)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:12.534288 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e859550-1578-4647-b340-e2de7d09438e None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:12.536093 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 169/327] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Aug 10 08:23:12 2022] GET /identity/v3/users/a49ae0c00a214b9c8d12294a561d6ce2 => generated 293 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:12.543301 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aa12f4-882d-45a5-9554-990a9b519548 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:12.623410 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec23c1f3-208d-4079-af15-ab4280e00eaa None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:12.623752 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec23c1f3-208d-4079-af15-ab4280e00eaa None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:12.624004 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec23c1f3-208d-4079-af15-ab4280e00eaa None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:12.625665 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 170/328] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:23:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 08:23:12.630936 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b6222a-fea7-4af1-8371-da208cca15bd None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:12.630936 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b6222a-fea7-4af1-8371-da208cca15bd None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:12.631501 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b6222a-fea7-4af1-8371-da208cca15bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:12.651530 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-18b6222a-fea7-4af1-8371-da208cca15bd None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:23:12.652772 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-18b6222a-fea7-4af1-8371-da208cca15bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:23:12.665674 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aa12f4-882d-45a5-9554-990a9b519548 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:12.684749 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:12.711147 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224d2e1f-7308-425f-b53d-e44478716cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:12.712177 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224d2e1f-7308-425f-b53d-e44478716cad None admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:12.712177 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224d2e1f-7308-425f-b53d-e44478716cad None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:12.712443 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224d2e1f-7308-425f-b53d-e44478716cad None admin] PATH_INFO: `/v3/system/users/a49ae0c00a214b9c8d12294a561d6ce2/roles/7237967510e447498d7b37ff7d0ea7ba` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:12.713964 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224d2e1f-7308-425f-b53d-e44478716cad None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=a49ae0c00a214b9c8d12294a561d6ce2, role_id=7237967510e447498d7b37ff7d0ea7ba)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:12.715387 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224d2e1f-7308-425f-b53d-e44478716cad None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:12.743834 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 171/329] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:23:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 08:23:12.768585 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b6222a-fea7-4af1-8371-da208cca15bd None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:12.793556 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 159/330] 10.222.0.57 () {58 vars in 1334 bytes} [Wed Aug 10 08:23:12 2022] PUT /identity/v3/system/users/a49ae0c00a214b9c8d12294a561d6ce2/roles/7237967510e447498d7b37ff7d0ea7ba => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:23:12.843548 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:12.901248 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b6222a-fea7-4af1-8371-da208cca15bd None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:12.940263 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003d264b-756a-4e84-8b97-1036dfd0d1cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:12.940933 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003d264b-756a-4e84-8b97-1036dfd0d1cb None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:12.941212 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003d264b-756a-4e84-8b97-1036dfd0d1cb None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:12.941480 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003d264b-756a-4e84-8b97-1036dfd0d1cb None admin] PATH_INFO: `/v3/roles/member` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:12.955626 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003d264b-756a-4e84-8b97-1036dfd0d1cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 08:23:12.961925 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-003d264b-756a-4e84-8b97-1036dfd0d1cb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 08:23:12.961925 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:12.961925 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:12.961925 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:12.961925 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:12.961925 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:12.961925 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:12.961925 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:12.961925 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:12.961925 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:12.961925 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:12.961925 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:12.961925 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 08:23:12.961925 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 08:23:12.961925 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 08:23:12.961925 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 08:23:12.962890 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 08:23:12.962890 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 08:23:12.962890 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 08:23:12.962890 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 08:23:12.962890 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:12.962890 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:12.962890 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:12.962890 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:12.962890 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:12.962890 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:12.962890 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:12.962890 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:12.962890 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:12.962890 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:12.962890 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:12.962890 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:12.963772 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:12.963772 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:12.963772 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:12.963772 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:12.963772 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 08:23:12.963772 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 08:23:12.963772 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 08:23:12.963772 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 08:23:12.963772 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 08:23:12.963772 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 08:23:12.963772 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 08:23:12.963772 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:23:12.963772 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 172/331] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Aug 10 08:23:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 08:23:12.970795 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224d2e1f-7308-425f-b53d-e44478716cad None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:13.093993 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224d2e1f-7308-425f-b53d-e44478716cad None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:13.135255 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bc8958-551e-4591-b9ff-88731e60e90d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:13.135874 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bc8958-551e-4591-b9ff-88731e60e90d None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:13.136154 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bc8958-551e-4591-b9ff-88731e60e90d None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:13.136426 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bc8958-551e-4591-b9ff-88731e60e90d None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:13.137127 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bc8958-551e-4591-b9ff-88731e60e90d None admin] RBAC: Adding query filter params (name=member) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:13.137429 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bc8958-551e-4591-b9ff-88731e60e90d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:13.138799 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bc8958-551e-4591-b9ff-88731e60e90d None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:13.148640 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 160/332] 10.222.0.57 () {56 vars in 1101 bytes} [Wed Aug 10 08:23:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:13.155617 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003d264b-756a-4e84-8b97-1036dfd0d1cb None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:13.278731 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003d264b-756a-4e84-8b97-1036dfd0d1cb None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:13.294792 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:13.316159 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c45973a-3207-45a7-8f6e-4efd82e449e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:13.317183 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c45973a-3207-45a7-8f6e-4efd82e449e9 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:13.317707 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c45973a-3207-45a7-8f6e-4efd82e449e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:13.318342 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c45973a-3207-45a7-8f6e-4efd82e449e9 None admin] PATH_INFO: `/v3/users/443e6b4e30d74256a01c04baa8a45e94` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:13.319779 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c45973a-3207-45a7-8f6e-4efd82e449e9 None admin] RBAC: Authorizing `identity:get_user(user_id=443e6b4e30d74256a01c04baa8a45e94)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:13.321599 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c45973a-3207-45a7-8f6e-4efd82e449e9 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:13.323672 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 173/333] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Aug 10 08:23:13 2022] GET /identity/v3/users/443e6b4e30d74256a01c04baa8a45e94 => generated 293 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:13.329851 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bc8958-551e-4591-b9ff-88731e60e90d None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:13.381399 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:13.470334 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bc8958-551e-4591-b9ff-88731e60e90d None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:13.523642 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8106c7a5-d2dc-48cf-97bb-4192fd16e4bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:13.524269 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8106c7a5-d2dc-48cf-97bb-4192fd16e4bd None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:13.524547 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8106c7a5-d2dc-48cf-97bb-4192fd16e4bd None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:13.524813 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8106c7a5-d2dc-48cf-97bb-4192fd16e4bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:13.525582 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8106c7a5-d2dc-48cf-97bb-4192fd16e4bd None admin] RBAC: Adding query filter params (role.id=44b64db484f24a46bf0b7ced46b519bf, user.id=443e6b4e30d74256a01c04baa8a45e94, scope.system=all) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:13.525886 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8106c7a5-d2dc-48cf-97bb-4192fd16e4bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:13.527428 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8106c7a5-d2dc-48cf-97bb-4192fd16e4bd None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:13.545401 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 161/334] 10.222.0.57 () {56 vars in 1308 bytes} [Wed Aug 10 08:23:13 2022] GET /identity/v3/role_assignments?user.id=443e6b4e30d74256a01c04baa8a45e94&scope.system=all&role.id=44b64db484f24a46bf0b7ced46b519bf => generated 510 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:14.805689 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a9faf6-3a3b-45e6-aea1-7684848069f7 None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:14.806200 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a9faf6-3a3b-45e6-aea1-7684848069f7 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:14.806305 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a9faf6-3a3b-45e6-aea1-7684848069f7 None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:14.808032 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 174/335] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:23:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:23:14.813125 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82485c9-8307-4a84-a3e1-43023017c502 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:14.813555 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82485c9-8307-4a84-a3e1-43023017c502 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:14.813728 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82485c9-8307-4a84-a3e1-43023017c502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:14.837558 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-b82485c9-8307-4a84-a3e1-43023017c502 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:23:14.838774 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-b82485c9-8307-4a84-a3e1-43023017c502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:23:14.922537 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 162/336] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:23:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 08:23:14.968771 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a9faf6-3a3b-45e6-aea1-7684848069f7 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:15.102059 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a9faf6-3a3b-45e6-aea1-7684848069f7 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:15.144470 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb0a8ac-bf1c-41bb-938a-3c0f5f49bf39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:15.145225 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb0a8ac-bf1c-41bb-938a-3c0f5f49bf39 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:15.145509 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb0a8ac-bf1c-41bb-938a-3c0f5f49bf39 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:15.145842 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb0a8ac-bf1c-41bb-938a-3c0f5f49bf39 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:15.157017 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb0a8ac-bf1c-41bb-938a-3c0f5f49bf39 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 08:23:15.162523 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-3bb0a8ac-bf1c-41bb-938a-3c0f5f49bf39 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 08:23:15.162523 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:15.162523 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:15.162523 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:15.162523 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:15.162523 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:15.162523 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:15.162523 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:15.162523 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:15.162523 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:15.162523 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:15.162523 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:15.162523 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 08:23:15.162523 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 08:23:15.162523 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 08:23:15.162523 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 08:23:15.163408 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 08:23:15.163408 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 08:23:15.163408 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 08:23:15.163408 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 08:23:15.163408 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:15.163408 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:15.163408 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:15.163408 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:15.163408 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:15.163408 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:15.163408 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:15.163408 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:15.163408 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:15.163408 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:15.163408 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:15.163408 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:15.164241 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:15.164241 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:15.164241 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:15.164241 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:15.164241 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 08:23:15.164241 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 08:23:15.164241 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 08:23:15.164241 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 08:23:15.164241 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 08:23:15.164241 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 08:23:15.164241 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 08:23:15.164241 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:23:15.164241 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 175/337] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Aug 10 08:23:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:15.170369 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82485c9-8307-4a84-a3e1-43023017c502 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:15.210213 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:15.242323 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d63c3a6-b1a3-4fbd-8607-03fa2e608b78 None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:15.242871 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d63c3a6-b1a3-4fbd-8607-03fa2e608b78 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:15.243295 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d63c3a6-b1a3-4fbd-8607-03fa2e608b78 None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:15.244788 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 176/338] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:23:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:23:15.255851 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11fbd33-c115-400e-b8bf-01b7c0773571 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:15.256396 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11fbd33-c115-400e-b8bf-01b7c0773571 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:15.256870 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11fbd33-c115-400e-b8bf-01b7c0773571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:15.287207 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-a11fbd33-c115-400e-b8bf-01b7c0773571 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:23:15.288768 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-a11fbd33-c115-400e-b8bf-01b7c0773571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:23:15.331213 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82485c9-8307-4a84-a3e1-43023017c502 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:15.344037 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:15.382614 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 177/339] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:23:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 08:23:15.413486 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24747da-7de6-47f0-b0f3-0b1424aaff48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:15.414423 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24747da-7de6-47f0-b0f3-0b1424aaff48 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:15.415648 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11fbd33-c115-400e-b8bf-01b7c0773571 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:15.416457 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24747da-7de6-47f0-b0f3-0b1424aaff48 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:15.416737 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24747da-7de6-47f0-b0f3-0b1424aaff48 None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:15.417472 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24747da-7de6-47f0-b0f3-0b1424aaff48 None admin] RBAC: Adding query filter params (name=member) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:15.417777 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24747da-7de6-47f0-b0f3-0b1424aaff48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:15.419360 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24747da-7de6-47f0-b0f3-0b1424aaff48 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:15.435515 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 163/340] 10.222.0.57 () {56 vars in 1101 bytes} [Wed Aug 10 08:23:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:15.441858 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24747da-7de6-47f0-b0f3-0b1424aaff48 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:15.562386 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11fbd33-c115-400e-b8bf-01b7c0773571 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:15.589093 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24747da-7de6-47f0-b0f3-0b1424aaff48 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:15.590924 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:15.622175 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65df6135-c0f8-4f4f-b692-62cb8e4be0f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:15.622818 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65df6135-c0f8-4f4f-b692-62cb8e4be0f3 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:15.623093 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65df6135-c0f8-4f4f-b692-62cb8e4be0f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:15.623361 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65df6135-c0f8-4f4f-b692-62cb8e4be0f3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:15.641818 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f8196d-ba01-4a48-ac72-03b98d018cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:15.642371 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f8196d-ba01-4a48-ac72-03b98d018cef None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:15.642481 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65df6135-c0f8-4f4f-b692-62cb8e4be0f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 08:23:15.642645 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f8196d-ba01-4a48-ac72-03b98d018cef None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:15.642915 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f8196d-ba01-4a48-ac72-03b98d018cef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:15.648245 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-65df6135-c0f8-4f4f-b692-62cb8e4be0f3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 08:23:15.648245 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:15.648245 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:15.648245 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:15.648245 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:15.648245 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:15.648245 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:15.648245 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:15.648245 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:15.648245 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:15.648245 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:15.648245 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:15.648245 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 08:23:15.648245 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 08:23:15.648245 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 08:23:15.648245 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 08:23:15.649242 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 08:23:15.649242 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 08:23:15.649242 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 08:23:15.649242 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 08:23:15.649242 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:15.649242 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:15.649242 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:15.649242 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:15.649242 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:15.649242 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:15.649242 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:15.649242 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:15.649242 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:15.649242 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:15.649242 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:15.649242 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:15.650072 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:15.650072 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:15.650072 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:15.650072 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:15.650072 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 08:23:15.650072 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 08:23:15.650072 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 08:23:15.650072 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 08:23:15.650072 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 08:23:15.650072 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 08:23:15.650072 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 08:23:15.650072 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:23:15.661630 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 178/341] 10.222.0.57 () {56 vars in 1114 bytes} [Wed Aug 10 08:23:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:15.674031 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65df6135-c0f8-4f4f-b692-62cb8e4be0f3 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:15.733209 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:15.752143 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f8196d-ba01-4a48-ac72-03b98d018cef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:15.756946 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f8196d-ba01-4a48-ac72-03b98d018cef None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:15.813470 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65df6135-c0f8-4f4f-b692-62cb8e4be0f3 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:15.862199 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:15.865828 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aceecc6-ff30-4717-8c24-46f490b6dfbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:15.866451 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aceecc6-ff30-4717-8c24-46f490b6dfbd None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:15.866731 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aceecc6-ff30-4717-8c24-46f490b6dfbd None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:15.867006 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aceecc6-ff30-4717-8c24-46f490b6dfbd None admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:15.867688 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aceecc6-ff30-4717-8c24-46f490b6dfbd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:15.867992 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aceecc6-ff30-4717-8c24-46f490b6dfbd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:15.869401 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aceecc6-ff30-4717-8c24-46f490b6dfbd None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:15.869527 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 164/342] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Aug 10 08:23:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 08:23:15.871148 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2fbbf3-e1a7-423a-ba6e-1ae891e3ad15 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:15.871433 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2fbbf3-e1a7-423a-ba6e-1ae891e3ad15 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:15.871708 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2fbbf3-e1a7-423a-ba6e-1ae891e3ad15 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:15.872990 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 165/343] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:23:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:23:15.877822 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2fbbf3-e1a7-423a-ba6e-1ae891e3ad15 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:15.878670 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 179/344] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Aug 10 08:23:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:15.884252 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7de58a-eafd-46bc-b3fa-6dc5114403e9 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:15.884555 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7de58a-eafd-46bc-b3fa-6dc5114403e9 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:15.884830 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7de58a-eafd-46bc-b3fa-6dc5114403e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:15.925932 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-4d7de58a-eafd-46bc-b3fa-6dc5114403e9 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:23:15.927153 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-4d7de58a-eafd-46bc-b3fa-6dc5114403e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:23:15.986946 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2fbbf3-e1a7-423a-ba6e-1ae891e3ad15 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:16.024812 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dac28bb-71b0-497d-9e44-6ad053c7cf6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:16.025489 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dac28bb-71b0-497d-9e44-6ad053c7cf6b None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:16.025866 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dac28bb-71b0-497d-9e44-6ad053c7cf6b None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:16.026142 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dac28bb-71b0-497d-9e44-6ad053c7cf6b None admin] PATH_INFO: `/v3/projects/0f2b0269b8b24890b55cca8078954ba1` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:16.027374 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dac28bb-71b0-497d-9e44-6ad053c7cf6b None admin] RBAC: Authorizing `identity:get_project(project_id=0f2b0269b8b24890b55cca8078954ba1)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:16.029017 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dac28bb-71b0-497d-9e44-6ad053c7cf6b None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:16.030957 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 166/345] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Aug 10 08:23:15 2022] GET /identity/v3/projects/0f2b0269b8b24890b55cca8078954ba1 => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:16.031728 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dac28bb-71b0-497d-9e44-6ad053c7cf6b None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:16.046606 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 180/346] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:23:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 08:23:16.047465 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7de58a-eafd-46bc-b3fa-6dc5114403e9 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:16.093552 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:16.116138 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:16.163259 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dac28bb-71b0-497d-9e44-6ad053c7cf6b None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:16.165870 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7de58a-eafd-46bc-b3fa-6dc5114403e9 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:16.201937 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb202368-5fc6-416b-bb28-1f5df94b4495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:16.202478 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb202368-5fc6-416b-bb28-1f5df94b4495 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:16.202763 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb202368-5fc6-416b-bb28-1f5df94b4495 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:16.203037 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb202368-5fc6-416b-bb28-1f5df94b4495 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:16.204261 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49997279-111b-48a9-9b4b-444d32d397b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:16.204900 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49997279-111b-48a9-9b4b-444d32d397b9 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:16.205199 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49997279-111b-48a9-9b4b-444d32d397b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:16.205465 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49997279-111b-48a9-9b4b-444d32d397b9 None admin] PATH_INFO: `/v3/groups/db43b5c3821f4fda98bfe4343e15a822` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:16.212949 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49997279-111b-48a9-9b4b-444d32d397b9 None admin] RBAC: Authorizing `identity:get_group(group_id=db43b5c3821f4fda98bfe4343e15a822)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:16.214405 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49997279-111b-48a9-9b4b-444d32d397b9 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:16.216381 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 181/347] 10.222.0.57 () {56 vars in 1180 bytes} [Wed Aug 10 08:23:16 2022] GET /identity/v3/groups/db43b5c3821f4fda98bfe4343e15a822 => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:16.219892 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ae6ce1-74f9-4d36-a17b-905d5c6a82f4 None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:16.220170 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ae6ce1-74f9-4d36-a17b-905d5c6a82f4 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:16.220440 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ae6ce1-74f9-4d36-a17b-905d5c6a82f4 None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:16.221711 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 182/348] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:23:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:23:16.222413 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ae6ce1-74f9-4d36-a17b-905d5c6a82f4 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:16.282332 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb202368-5fc6-416b-bb28-1f5df94b4495 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:16.284039 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb202368-5fc6-416b-bb28-1f5df94b4495 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:16.319441 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:16.333426 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ae6ce1-74f9-4d36-a17b-905d5c6a82f4 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:16.348698 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:16.371128 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cc035c-48c2-4617-89cf-387a620747c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:16.371563 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cc035c-48c2-4617-89cf-387a620747c8 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:16.371896 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cc035c-48c2-4617-89cf-387a620747c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:16.372168 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cc035c-48c2-4617-89cf-387a620747c8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:16.385359 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cc035c-48c2-4617-89cf-387a620747c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 08:23:16.388375 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-93cc035c-48c2-4617-89cf-387a620747c8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 08:23:16.388375 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:16.388375 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:16.388375 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:16.388375 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:16.388375 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:16.388375 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:16.388375 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:16.388375 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:16.388375 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:16.388375 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:16.388375 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:16.388375 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 08:23:16.388375 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 08:23:16.388375 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 08:23:16.388375 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 08:23:16.389538 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 08:23:16.389538 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 08:23:16.389538 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 08:23:16.389538 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 08:23:16.389538 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:16.389538 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:16.389538 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:16.389538 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:16.389538 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:16.389538 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:16.389538 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:16.389538 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:16.389538 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:16.389538 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:16.389538 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:16.389538 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:16.390631 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:16.390631 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:16.390631 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:16.390631 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:16.390631 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 08:23:16.390631 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 08:23:16.390631 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 08:23:16.390631 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 08:23:16.390631 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 08:23:16.390631 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 08:23:16.390631 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 08:23:16.390631 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:23:16.390631 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 183/349] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Aug 10 08:23:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:16.391422 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cc035c-48c2-4617-89cf-387a620747c8 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:16.419044 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 167/350] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Aug 10 08:23:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 08:23:16.420749 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fbd52c-9903-4277-bfa0-5485fe1bc276 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:16.421221 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fbd52c-9903-4277-bfa0-5485fe1bc276 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:16.421640 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fbd52c-9903-4277-bfa0-5485fe1bc276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:16.444150 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-18fbd52c-9903-4277-bfa0-5485fe1bc276 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:23:16.445703 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-18fbd52c-9903-4277-bfa0-5485fe1bc276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:23:16.523915 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cc035c-48c2-4617-89cf-387a620747c8 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:16.537177 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 168/351] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:23:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 08:23:16.537825 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fbd52c-9903-4277-bfa0-5485fe1bc276 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:16.593389 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064e56ac-58d2-4f70-8411-d101b426136d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:16.594150 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064e56ac-58d2-4f70-8411-d101b426136d None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:16.594527 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064e56ac-58d2-4f70-8411-d101b426136d None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:16.594897 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064e56ac-58d2-4f70-8411-d101b426136d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:16.595735 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064e56ac-58d2-4f70-8411-d101b426136d None admin] RBAC: Adding query filter params (group.id=db43b5c3821f4fda98bfe4343e15a822, role.id=44b64db484f24a46bf0b7ced46b519bf, scope.project.id=0f2b0269b8b24890b55cca8078954ba1) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:16.596145 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064e56ac-58d2-4f70-8411-d101b426136d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:16.597720 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064e56ac-58d2-4f70-8411-d101b426136d None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:16.605531 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 184/352] 10.222.0.57 () {56 vars in 1376 bytes} [Wed Aug 10 08:23:16 2022] GET /identity/v3/role_assignments?group.id=db43b5c3821f4fda98bfe4343e15a822&scope.project.id=0f2b0269b8b24890b55cca8078954ba1&role.id=44b64db484f24a46bf0b7ced46b519bf => generated 260 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:16.606816 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064e56ac-58d2-4f70-8411-d101b426136d None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:16.645121 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:16.657637 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:16.671769 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fbd52c-9903-4277-bfa0-5485fe1bc276 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:16.716619 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce730891-993a-4dec-9d44-ebfb93656dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:16.719884 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce730891-993a-4dec-9d44-ebfb93656dd9 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:16.720421 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce730891-993a-4dec-9d44-ebfb93656dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:16.721783 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce730891-993a-4dec-9d44-ebfb93656dd9 None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:16.722668 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce730891-993a-4dec-9d44-ebfb93656dd9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:16.724601 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce730891-993a-4dec-9d44-ebfb93656dd9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:16.726414 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce730891-993a-4dec-9d44-ebfb93656dd9 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:16.746241 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 169/353] 10.222.0.57 () {56 vars in 1101 bytes} [Wed Aug 10 08:23:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:16.747214 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce730891-993a-4dec-9d44-ebfb93656dd9 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:16.821798 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064e56ac-58d2-4f70-8411-d101b426136d None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:16.865393 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0600f123-e3ac-41b2-9dc7-4eb6761c3e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:16.866928 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0600f123-e3ac-41b2-9dc7-4eb6761c3e8f None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:16.867391 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0600f123-e3ac-41b2-9dc7-4eb6761c3e8f None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:16.867842 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0600f123-e3ac-41b2-9dc7-4eb6761c3e8f None admin] PATH_INFO: `/v3/projects/0f2b0269b8b24890b55cca8078954ba1` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:16.869743 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0600f123-e3ac-41b2-9dc7-4eb6761c3e8f None admin] RBAC: Authorizing `identity:get_project(project_id=0f2b0269b8b24890b55cca8078954ba1)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:16.872566 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0600f123-e3ac-41b2-9dc7-4eb6761c3e8f None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:16.874760 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 185/354] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Aug 10 08:23:16 2022] GET /identity/v3/projects/0f2b0269b8b24890b55cca8078954ba1 => generated 298 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:16.875779 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0600f123-e3ac-41b2-9dc7-4eb6761c3e8f None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:16.920837 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce730891-993a-4dec-9d44-ebfb93656dd9 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:16.959189 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ddb9b5-cbaa-4949-bae2-9191c65a28fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:16.959850 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ddb9b5-cbaa-4949-bae2-9191c65a28fd None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:16.960137 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ddb9b5-cbaa-4949-bae2-9191c65a28fd None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:16.960407 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ddb9b5-cbaa-4949-bae2-9191c65a28fd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:16.971871 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ddb9b5-cbaa-4949-bae2-9191c65a28fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 08:23:16.979181 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-e5ddb9b5-cbaa-4949-bae2-9191c65a28fd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 08:23:16.979181 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:16.979181 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:16.979181 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:16.979181 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:16.979181 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:16.979181 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:16.979181 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:16.979181 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:16.979181 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:16.979181 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:16.979181 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:16.979181 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 08:23:16.979181 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 08:23:16.979181 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 08:23:16.979181 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 08:23:16.980146 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 08:23:16.980146 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 08:23:16.980146 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 08:23:16.980146 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 08:23:16.980146 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:16.980146 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:16.980146 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:16.980146 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:16.980146 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:16.980146 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:16.980146 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:16.980146 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:16.980146 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:16.980146 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:16.980146 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:16.980146 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:16.981114 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:16.981114 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:16.981114 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:16.981114 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:16.981114 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 08:23:16.981114 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 08:23:16.981114 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 08:23:16.981114 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 08:23:16.981114 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 08:23:16.981114 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 08:23:16.981114 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 08:23:16.981114 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:23:16.981114 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 170/355] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Aug 10 08:23:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:16.981114 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ddb9b5-cbaa-4949-bae2-9191c65a28fd None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:16.987285 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:17.001197 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:17.011489 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0600f123-e3ac-41b2-9dc7-4eb6761c3e8f None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:17.053431 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cea440-53f8-4d06-9ca9-cee5829a064c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:17.054328 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cea440-53f8-4d06-9ca9-cee5829a064c None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:17.054742 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cea440-53f8-4d06-9ca9-cee5829a064c None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:17.055154 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cea440-53f8-4d06-9ca9-cee5829a064c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:17.110015 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ddb9b5-cbaa-4949-bae2-9191c65a28fd None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:17.130264 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cea440-53f8-4d06-9ca9-cee5829a064c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:17.132278 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cea440-53f8-4d06-9ca9-cee5829a064c None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:17.150000 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471b70d0-462c-40dc-8f2e-124e48543629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:17.150689 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471b70d0-462c-40dc-8f2e-124e48543629 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:17.151068 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471b70d0-462c-40dc-8f2e-124e48543629 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:17.151461 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471b70d0-462c-40dc-8f2e-124e48543629 None admin] PATH_INFO: `/v3/groups/db43b5c3821f4fda98bfe4343e15a822` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:17.152857 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471b70d0-462c-40dc-8f2e-124e48543629 None admin] RBAC: Authorizing `identity:get_group(group_id=db43b5c3821f4fda98bfe4343e15a822)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:17.154520 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471b70d0-462c-40dc-8f2e-124e48543629 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:17.156375 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 171/356] 10.222.0.57 () {56 vars in 1180 bytes} [Wed Aug 10 08:23:16 2022] GET /identity/v3/groups/db43b5c3821f4fda98bfe4343e15a822 => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:17.157570 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfad0ad5-e7d9-4d71-be99-da09281ad41b None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:17.157953 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfad0ad5-e7d9-4d71-be99-da09281ad41b None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:17.158383 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfad0ad5-e7d9-4d71-be99-da09281ad41b None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:17.159694 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 172/357] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:23:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:23:17.160565 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfad0ad5-e7d9-4d71-be99-da09281ad41b None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:17.229029 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:17.252251 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 186/358] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Aug 10 08:23:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 08:23:17.257967 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cea440-53f8-4d06-9ca9-cee5829a064c None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:17.258484 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:17.285261 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfad0ad5-e7d9-4d71-be99-da09281ad41b None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:17.327799 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79f8fa4-f049-41a2-bd96-24e240fac7bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:17.328664 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79f8fa4-f049-41a2-bd96-24e240fac7bc None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:17.329909 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79f8fa4-f049-41a2-bd96-24e240fac7bc None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:17.329909 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79f8fa4-f049-41a2-bd96-24e240fac7bc None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:17.330094 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79f8fa4-f049-41a2-bd96-24e240fac7bc None admin] RBAC: Adding query filter params (name=reader) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:17.330337 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79f8fa4-f049-41a2-bd96-24e240fac7bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:17.331707 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79f8fa4-f049-41a2-bd96-24e240fac7bc None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:17.345081 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 173/359] 10.222.0.57 () {56 vars in 1101 bytes} [Wed Aug 10 08:23:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:17.345081 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724f977d-67b9-4d75-a17a-27b0327c28c3 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:17.345081 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724f977d-67b9-4d75-a17a-27b0327c28c3 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:17.345081 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724f977d-67b9-4d75-a17a-27b0327c28c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:17.365693 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-724f977d-67b9-4d75-a17a-27b0327c28c3 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:23:17.366855 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-724f977d-67b9-4d75-a17a-27b0327c28c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:23:17.413927 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cea440-53f8-4d06-9ca9-cee5829a064c None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:17.463662 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb49bfce-734c-4981-bc89-47905ae7b2c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:17.464279 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb49bfce-734c-4981-bc89-47905ae7b2c6 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:17.464569 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb49bfce-734c-4981-bc89-47905ae7b2c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:17.464825 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb49bfce-734c-4981-bc89-47905ae7b2c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:17.465586 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb49bfce-734c-4981-bc89-47905ae7b2c6 None admin] RBAC: Adding query filter params (group.id=db43b5c3821f4fda98bfe4343e15a822, role.id=ccfc8d3fdf2d458299df29001278930f, scope.project.id=0f2b0269b8b24890b55cca8078954ba1) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:17.465889 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb49bfce-734c-4981-bc89-47905ae7b2c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:17.467357 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb49bfce-734c-4981-bc89-47905ae7b2c6 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:17.471632 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 174/360] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:23:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 08:23:17.472724 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724f977d-67b9-4d75-a17a-27b0327c28c3 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:17.473721 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 187/361] 10.222.0.57 () {56 vars in 1376 bytes} [Wed Aug 10 08:23:17 2022] GET /identity/v3/role_assignments?group.id=db43b5c3821f4fda98bfe4343e15a822&scope.project.id=0f2b0269b8b24890b55cca8078954ba1&role.id=ccfc8d3fdf2d458299df29001278930f => generated 260 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:17.475810 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad7675f-c7f2-4ab5-aaff-f93429eee08d None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:17.476129 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad7675f-c7f2-4ab5-aaff-f93429eee08d None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:17.476499 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad7675f-c7f2-4ab5-aaff-f93429eee08d None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:17.477888 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 188/362] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:23:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:23:17.478731 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad7675f-c7f2-4ab5-aaff-f93429eee08d None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:17.561322 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:17.581773 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:17.616491 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724f977d-67b9-4d75-a17a-27b0327c28c3 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:17.633418 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad7675f-c7f2-4ab5-aaff-f93429eee08d None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:17.657434 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa53407a-ec8c-4ea4-b7f1-42d74348763e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:17.658356 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa53407a-ec8c-4ea4-b7f1-42d74348763e None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:17.658818 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa53407a-ec8c-4ea4-b7f1-42d74348763e None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:17.659262 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa53407a-ec8c-4ea4-b7f1-42d74348763e None admin] PATH_INFO: `/v3/projects/201dfacf93654f63a4732891e33d5039` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:17.660583 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa53407a-ec8c-4ea4-b7f1-42d74348763e None admin] RBAC: Authorizing `identity:get_project(project_id=201dfacf93654f63a4732891e33d5039)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:17.662844 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa53407a-ec8c-4ea4-b7f1-42d74348763e None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:17.664693 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 175/363] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Aug 10 08:23:17 2022] GET /identity/v3/projects/201dfacf93654f63a4732891e33d5039 => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:17.666020 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8d817b-a67e-4c49-91b7-15c8f1a9c562 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:17.666480 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8d817b-a67e-4c49-91b7-15c8f1a9c562 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:17.666924 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8d817b-a67e-4c49-91b7-15c8f1a9c562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:17.675327 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90cf43b-79f5-4426-a33b-f13689983858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:17.675745 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90cf43b-79f5-4426-a33b-f13689983858 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:17.676086 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90cf43b-79f5-4426-a33b-f13689983858 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:17.676360 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90cf43b-79f5-4426-a33b-f13689983858 None admin] PATH_INFO: `/v3/users/a49ae0c00a214b9c8d12294a561d6ce2` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:17.677622 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90cf43b-79f5-4426-a33b-f13689983858 None admin] RBAC: Authorizing `identity:get_user(user_id=a49ae0c00a214b9c8d12294a561d6ce2)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:17.679070 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90cf43b-79f5-4426-a33b-f13689983858 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:17.681766 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 189/364] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Aug 10 08:23:17 2022] GET /identity/v3/users/a49ae0c00a214b9c8d12294a561d6ce2 => generated 293 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:17.682597 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90cf43b-79f5-4426-a33b-f13689983858 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:17.688079 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-bd8d817b-a67e-4c49-91b7-15c8f1a9c562 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:23:17.689548 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-bd8d817b-a67e-4c49-91b7-15c8f1a9c562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:23:17.761948 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 176/365] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:23:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 08:23:17.763125 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8d817b-a67e-4c49-91b7-15c8f1a9c562 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:17.791733 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90cf43b-79f5-4426-a33b-f13689983858 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:17.806601 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:17.830416 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b9e36c-238d-4cfc-8336-e6d7c724a2e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:17.831073 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b9e36c-238d-4cfc-8336-e6d7c724a2e1 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:17.831317 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b9e36c-238d-4cfc-8336-e6d7c724a2e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:17.831590 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b9e36c-238d-4cfc-8336-e6d7c724a2e1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:17.836902 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:17.842556 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b9e36c-238d-4cfc-8336-e6d7c724a2e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 08:23:17.847946 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-f6b9e36c-238d-4cfc-8336-e6d7c724a2e1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 08:23:17.847946 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:17.847946 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:17.847946 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:17.847946 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:17.847946 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:17.847946 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:17.847946 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:17.847946 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:17.847946 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:17.847946 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:17.847946 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:17.847946 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 08:23:17.847946 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 08:23:17.847946 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 08:23:17.847946 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 08:23:17.851596 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 08:23:17.851596 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 08:23:17.851596 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 08:23:17.851596 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 08:23:17.851596 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:17.851596 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:17.851596 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:17.851596 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:17.851596 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:17.851596 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:17.851596 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:17.851596 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:17.851596 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:17.851596 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:17.851596 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:17.851596 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:17.852444 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:17.852444 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:17.852444 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:17.852444 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:17.852444 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 08:23:17.852444 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 08:23:17.852444 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 08:23:17.852444 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 08:23:17.852444 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 08:23:17.852444 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 08:23:17.852444 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 08:23:17.852444 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:23:17.852444 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 190/366] 10.222.0.57 () {56 vars in 1114 bytes} [Wed Aug 10 08:23:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:17.852444 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b9e36c-238d-4cfc-8336-e6d7c724a2e1 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:17.877155 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8d817b-a67e-4c49-91b7-15c8f1a9c562 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:17.915138 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d7ec83-38ff-4280-9d8c-49368bace5dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:17.916021 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d7ec83-38ff-4280-9d8c-49368bace5dc None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:17.916445 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d7ec83-38ff-4280-9d8c-49368bace5dc None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:17.916860 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d7ec83-38ff-4280-9d8c-49368bace5dc None admin] PATH_INFO: `/v3/groups/db43b5c3821f4fda98bfe4343e15a822` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:17.918344 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d7ec83-38ff-4280-9d8c-49368bace5dc None admin] RBAC: Authorizing `identity:get_group(group_id=db43b5c3821f4fda98bfe4343e15a822)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:17.919951 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d7ec83-38ff-4280-9d8c-49368bace5dc None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:17.922019 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 177/367] 10.222.0.57 () {56 vars in 1180 bytes} [Wed Aug 10 08:23:17 2022] GET /identity/v3/groups/db43b5c3821f4fda98bfe4343e15a822 => generated 226 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:17.923368 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d7ec83-38ff-4280-9d8c-49368bace5dc None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:17.964188 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b9e36c-238d-4cfc-8336-e6d7c724a2e1 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:18.002191 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a851f3-e454-408a-a5aa-b725936a6a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:18.002844 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a851f3-e454-408a-a5aa-b725936a6a9e None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:18.003129 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a851f3-e454-408a-a5aa-b725936a6a9e None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:18.003403 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a851f3-e454-408a-a5aa-b725936a6a9e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:18.004139 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a851f3-e454-408a-a5aa-b725936a6a9e None admin] RBAC: Adding query filter params (role.id=7237967510e447498d7b37ff7d0ea7ba, user.id=a49ae0c00a214b9c8d12294a561d6ce2, scope.system=all) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:18.004446 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a851f3-e454-408a-a5aa-b725936a6a9e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:18.007014 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a851f3-e454-408a-a5aa-b725936a6a9e None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:18.019895 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 191/368] 10.222.0.57 () {56 vars in 1308 bytes} [Wed Aug 10 08:23:17 2022] GET /identity/v3/role_assignments?user.id=a49ae0c00a214b9c8d12294a561d6ce2&scope.system=all&role.id=7237967510e447498d7b37ff7d0ea7ba => generated 510 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:18.020583 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a851f3-e454-408a-a5aa-b725936a6a9e None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:18.035137 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d7ec83-38ff-4280-9d8c-49368bace5dc None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:18.058060 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:18.080061 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd1e523-66c1-41c2-b53f-959fe5d988c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:18.081057 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd1e523-66c1-41c2-b53f-959fe5d988c6 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:18.081516 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd1e523-66c1-41c2-b53f-959fe5d988c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:18.081962 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd1e523-66c1-41c2-b53f-959fe5d988c6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:18.093303 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:18.094134 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd1e523-66c1-41c2-b53f-959fe5d988c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 08:23:18.099228 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-1bd1e523-66c1-41c2-b53f-959fe5d988c6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 08:23:18.099228 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:18.099228 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:18.099228 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:18.099228 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:18.099228 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:18.099228 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:18.099228 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:18.099228 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:18.099228 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:18.099228 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:18.099228 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:18.099228 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 08:23:18.099228 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 08:23:18.099228 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 08:23:18.099228 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 08:23:18.100154 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 08:23:18.100154 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 08:23:18.100154 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 08:23:18.100154 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 08:23:18.100154 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:18.100154 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:18.100154 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:18.100154 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:18.100154 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:18.100154 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:18.100154 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:18.100154 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:18.100154 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:18.100154 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:18.100154 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:18.100154 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:18.101508 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:18.101508 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:18.101508 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:18.101508 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:18.101508 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 08:23:18.101508 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 08:23:18.101508 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 08:23:18.101508 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 08:23:18.101508 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 08:23:18.101508 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 08:23:18.101508 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 08:23:18.101508 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:23:18.109825 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 178/369] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Aug 10 08:23:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:18.109825 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd1e523-66c1-41c2-b53f-959fe5d988c6 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:18.139333 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a851f3-e454-408a-a5aa-b725936a6a9e None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:18.178781 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241f9110-81be-4dfd-a8f7-b3668ebde4ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:18.179406 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241f9110-81be-4dfd-a8f7-b3668ebde4ce None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:18.179685 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241f9110-81be-4dfd-a8f7-b3668ebde4ce None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:18.179966 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241f9110-81be-4dfd-a8f7-b3668ebde4ce None admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:18.180757 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241f9110-81be-4dfd-a8f7-b3668ebde4ce None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:18.180938 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241f9110-81be-4dfd-a8f7-b3668ebde4ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:18.182401 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241f9110-81be-4dfd-a8f7-b3668ebde4ce None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:18.192539 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 192/370] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Aug 10 08:23:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:18.193294 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241f9110-81be-4dfd-a8f7-b3668ebde4ce None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:18.226556 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd1e523-66c1-41c2-b53f-959fe5d988c6 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:18.289737 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44607d1-6534-40a9-b376-8f7d63872118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:18.290682 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44607d1-6534-40a9-b376-8f7d63872118 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:18.291195 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44607d1-6534-40a9-b376-8f7d63872118 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:18.291640 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44607d1-6534-40a9-b376-8f7d63872118 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:18.292640 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44607d1-6534-40a9-b376-8f7d63872118 None admin] RBAC: Adding query filter params (group.id=db43b5c3821f4fda98bfe4343e15a822, role.id=44b64db484f24a46bf0b7ced46b519bf, scope.project.id=201dfacf93654f63a4732891e33d5039) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:18.293170 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44607d1-6534-40a9-b376-8f7d63872118 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:18.294817 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44607d1-6534-40a9-b376-8f7d63872118 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:18.301108 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 179/371] 10.222.0.57 () {56 vars in 1376 bytes} [Wed Aug 10 08:23:18 2022] GET /identity/v3/role_assignments?group.id=db43b5c3821f4fda98bfe4343e15a822&scope.project.id=201dfacf93654f63a4732891e33d5039&role.id=44b64db484f24a46bf0b7ced46b519bf => generated 260 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:18.302297 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44607d1-6534-40a9-b376-8f7d63872118 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:18.303170 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:18.313722 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241f9110-81be-4dfd-a8f7-b3668ebde4ce None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:18.357110 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5e6fc0-7eea-4790-a15c-660101b8a5af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:18.358038 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5e6fc0-7eea-4790-a15c-660101b8a5af None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:18.358482 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5e6fc0-7eea-4790-a15c-660101b8a5af None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:18.358942 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5e6fc0-7eea-4790-a15c-660101b8a5af None admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:18.359776 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5e6fc0-7eea-4790-a15c-660101b8a5af None admin] RBAC: Adding query filter params (name=admin) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:18.360301 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5e6fc0-7eea-4790-a15c-660101b8a5af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:18.362162 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5e6fc0-7eea-4790-a15c-660101b8a5af None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:18.373069 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 193/372] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Aug 10 08:23:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:18.378695 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5e6fc0-7eea-4790-a15c-660101b8a5af None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:18.435562 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:18.480704 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44607d1-6534-40a9-b376-8f7d63872118 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:18.525279 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246d8a22-c26d-4f69-8bd1-7b69e542e059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:18.525952 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246d8a22-c26d-4f69-8bd1-7b69e542e059 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:18.526169 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246d8a22-c26d-4f69-8bd1-7b69e542e059 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:18.526423 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246d8a22-c26d-4f69-8bd1-7b69e542e059 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:18.528191 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5e6fc0-7eea-4790-a15c-660101b8a5af None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:18.575595 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81466530-e0fb-4635-a5af-5b50e5e343ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:18.576212 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81466530-e0fb-4635-a5af-5b50e5e343ea None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:18.576490 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81466530-e0fb-4635-a5af-5b50e5e343ea None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:18.576763 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81466530-e0fb-4635-a5af-5b50e5e343ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:18.603906 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246d8a22-c26d-4f69-8bd1-7b69e542e059 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:18.605561 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246d8a22-c26d-4f69-8bd1-7b69e542e059 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:18.610986 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:18.671730 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:18.674567 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81466530-e0fb-4635-a5af-5b50e5e343ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:18.676033 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81466530-e0fb-4635-a5af-5b50e5e343ea None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:18.716574 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 180/373] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Aug 10 08:23:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 08:23:18.722202 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246d8a22-c26d-4f69-8bd1-7b69e542e059 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:18.828403 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246d8a22-c26d-4f69-8bd1-7b69e542e059 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:18.830319 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 194/374] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Aug 10 08:23:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 08:23:18.837337 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81466530-e0fb-4635-a5af-5b50e5e343ea None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:18.869611 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ca9b3d-cb15-4ec8-b75f-7ce959981a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:18.870246 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ca9b3d-cb15-4ec8-b75f-7ce959981a45 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:18.870764 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ca9b3d-cb15-4ec8-b75f-7ce959981a45 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:18.871044 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ca9b3d-cb15-4ec8-b75f-7ce959981a45 None admin] PATH_INFO: `/v3/projects/201dfacf93654f63a4732891e33d5039` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:18.874089 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ca9b3d-cb15-4ec8-b75f-7ce959981a45 None admin] RBAC: Authorizing `identity:get_project(project_id=201dfacf93654f63a4732891e33d5039)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:18.875518 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ca9b3d-cb15-4ec8-b75f-7ce959981a45 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:18.877621 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 181/375] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Aug 10 08:23:18 2022] GET /identity/v3/projects/201dfacf93654f63a4732891e33d5039 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:18.882974 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ca9b3d-cb15-4ec8-b75f-7ce959981a45 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:18.904748 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:18.926001 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:18.974886 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81466530-e0fb-4635-a5af-5b50e5e343ea None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:19.017439 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ca9b3d-cb15-4ec8-b75f-7ce959981a45 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:19.018689 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d317f84-0948-41af-b8d5-284f57dcfa7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:19.019499 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d317f84-0948-41af-b8d5-284f57dcfa7c None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:19.020595 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d317f84-0948-41af-b8d5-284f57dcfa7c None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:19.021739 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d317f84-0948-41af-b8d5-284f57dcfa7c None admin] PATH_INFO: `/v3/projects/fe95c65d461a414fa25f77dc13eab47b` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:19.023001 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d317f84-0948-41af-b8d5-284f57dcfa7c None admin] RBAC: Authorizing `identity:get_project(project_id=fe95c65d461a414fa25f77dc13eab47b)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:19.024449 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d317f84-0948-41af-b8d5-284f57dcfa7c None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:19.026457 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 195/376] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Aug 10 08:23:18 2022] GET /identity/v3/projects/fe95c65d461a414fa25f77dc13eab47b => generated 344 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:23:19.031572 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d317f84-0948-41af-b8d5-284f57dcfa7c None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:19.101236 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8f5ae0-81fe-45eb-a901-5ce6e925b304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:19.101883 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8f5ae0-81fe-45eb-a901-5ce6e925b304 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:19.102160 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8f5ae0-81fe-45eb-a901-5ce6e925b304 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:19.102429 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8f5ae0-81fe-45eb-a901-5ce6e925b304 None admin] PATH_INFO: `/v3/groups/db43b5c3821f4fda98bfe4343e15a822` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:19.103738 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8f5ae0-81fe-45eb-a901-5ce6e925b304 None admin] RBAC: Authorizing `identity:get_group(group_id=db43b5c3821f4fda98bfe4343e15a822)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:19.107552 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8f5ae0-81fe-45eb-a901-5ce6e925b304 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:19.112332 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 182/377] 10.222.0.57 () {56 vars in 1180 bytes} [Wed Aug 10 08:23:18 2022] GET /identity/v3/groups/db43b5c3821f4fda98bfe4343e15a822 => generated 226 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:23:19.116943 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8f5ae0-81fe-45eb-a901-5ce6e925b304 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:19.200102 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d317f84-0948-41af-b8d5-284f57dcfa7c None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:19.238929 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:19.251685 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3e5519-deed-43c0-9d3c-e4c782a3567a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:19.252309 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3e5519-deed-43c0-9d3c-e4c782a3567a None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:19.252584 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3e5519-deed-43c0-9d3c-e4c782a3567a None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:19.252852 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3e5519-deed-43c0-9d3c-e4c782a3567a None admin] PATH_INFO: `/v3/groups/a38ec50e49fb4028871d1d6f28421e35` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:19.258219 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3e5519-deed-43c0-9d3c-e4c782a3567a None admin] RBAC: Authorizing `identity:get_group(group_id=a38ec50e49fb4028871d1d6f28421e35)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:19.259665 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3e5519-deed-43c0-9d3c-e4c782a3567a None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:19.259871 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:19.262330 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 196/378] 10.222.0.57 () {56 vars in 1180 bytes} [Wed Aug 10 08:23:19 2022] GET /identity/v3/groups/a38ec50e49fb4028871d1d6f28421e35 => generated 229 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:23:19.263502 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cf047e-aa13-41bb-9136-17d8467308e7 None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:19.268694 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cf047e-aa13-41bb-9136-17d8467308e7 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:19.268694 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cf047e-aa13-41bb-9136-17d8467308e7 None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:19.268694 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 197/379] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:23:19 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:23:19.268951 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8f5ae0-81fe-45eb-a901-5ce6e925b304 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:19.270091 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cf047e-aa13-41bb-9136-17d8467308e7 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:19.305902 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cdf298-644e-417d-a701-697893070359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:19.306723 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cdf298-644e-417d-a701-697893070359 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:19.307095 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cdf298-644e-417d-a701-697893070359 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:19.307466 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cdf298-644e-417d-a701-697893070359 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:19.308205 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cdf298-644e-417d-a701-697893070359 None admin] RBAC: Adding query filter params (group.id=db43b5c3821f4fda98bfe4343e15a822, role.id=ccfc8d3fdf2d458299df29001278930f, scope.project.id=201dfacf93654f63a4732891e33d5039) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:19.308515 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cdf298-644e-417d-a701-697893070359 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:19.310005 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cdf298-644e-417d-a701-697893070359 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:19.316023 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 183/380] 10.222.0.57 () {56 vars in 1376 bytes} [Wed Aug 10 08:23:19 2022] GET /identity/v3/role_assignments?group.id=db43b5c3821f4fda98bfe4343e15a822&scope.project.id=201dfacf93654f63a4732891e33d5039&role.id=ccfc8d3fdf2d458299df29001278930f => generated 260 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:19.317333 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42ca3a7-a81f-401e-b6f4-0bbf11d1e9a2 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:19.317651 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42ca3a7-a81f-401e-b6f4-0bbf11d1e9a2 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:19.317920 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42ca3a7-a81f-401e-b6f4-0bbf11d1e9a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:19.338269 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-a42ca3a7-a81f-401e-b6f4-0bbf11d1e9a2 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:23:19.339519 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-a42ca3a7-a81f-401e-b6f4-0bbf11d1e9a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:23:19.387983 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cf047e-aa13-41bb-9136-17d8467308e7 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:19.416678 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 184/381] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:23:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 08:23:19.424201 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fec12e-5bd9-4f69-9dfa-6788daf256e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:19.424820 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fec12e-5bd9-4f69-9dfa-6788daf256e6 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:19.425127 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fec12e-5bd9-4f69-9dfa-6788daf256e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:19.425393 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fec12e-5bd9-4f69-9dfa-6788daf256e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:19.426120 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fec12e-5bd9-4f69-9dfa-6788daf256e6 None admin] RBAC: Adding query filter params (group.id=a38ec50e49fb4028871d1d6f28421e35, role.id=ec3751692d9c4ecf81b4a71c936c46be, scope.project.id=fe95c65d461a414fa25f77dc13eab47b) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:19.426421 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fec12e-5bd9-4f69-9dfa-6788daf256e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:19.428501 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fec12e-5bd9-4f69-9dfa-6788daf256e6 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:19.434972 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 198/382] 10.222.0.57 () {56 vars in 1376 bytes} [Wed Aug 10 08:23:19 2022] GET /identity/v3/role_assignments?group.id=a38ec50e49fb4028871d1d6f28421e35&scope.project.id=fe95c65d461a414fa25f77dc13eab47b&role.id=ec3751692d9c4ecf81b4a71c936c46be => generated 260 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:19.441073 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42ca3a7-a81f-401e-b6f4-0bbf11d1e9a2 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:19.569417 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:19.579532 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42ca3a7-a81f-401e-b6f4-0bbf11d1e9a2 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:19.619236 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4141c321-fefe-4c7e-b7b7-26f9d92c8dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:19.619885 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4141c321-fefe-4c7e-b7b7-26f9d92c8dc1 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:19.620165 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4141c321-fefe-4c7e-b7b7-26f9d92c8dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:19.620439 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4141c321-fefe-4c7e-b7b7-26f9d92c8dc1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:19.633672 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4141c321-fefe-4c7e-b7b7-26f9d92c8dc1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 08:23:19.638485 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-4141c321-fefe-4c7e-b7b7-26f9d92c8dc1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 08:23:19.638485 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:19.638485 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:19.638485 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:19.638485 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:19.638485 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:19.638485 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:19.638485 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:19.638485 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:19.638485 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:19.638485 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:19.638485 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:19.638485 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 08:23:19.638485 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 08:23:19.638485 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 08:23:19.638485 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 08:23:19.639410 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 08:23:19.639410 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 08:23:19.639410 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 08:23:19.639410 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 08:23:19.639410 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:19.639410 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:19.639410 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:19.639410 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:19.639410 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:19.639410 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:19.639410 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:19.639410 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:19.639410 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:19.639410 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:19.639410 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:19.639410 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:19.640233 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:19.640233 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:19.640233 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:19.640233 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:19.640233 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 08:23:19.640233 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 08:23:19.640233 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 08:23:19.640233 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 08:23:19.640233 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 08:23:19.640233 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 08:23:19.640233 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 08:23:19.640233 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:23:19.640233 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 185/383] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Aug 10 08:23:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:19.646177 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fec12e-5bd9-4f69-9dfa-6788daf256e6 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:19.710369 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:19.768076 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fec12e-5bd9-4f69-9dfa-6788daf256e6 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:19.804859 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aebed8-3ef4-4dcb-9d8d-b1410330625f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:19.805573 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aebed8-3ef4-4dcb-9d8d-b1410330625f None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:19.805853 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aebed8-3ef4-4dcb-9d8d-b1410330625f None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:19.806121 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aebed8-3ef4-4dcb-9d8d-b1410330625f None admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:19.806841 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aebed8-3ef4-4dcb-9d8d-b1410330625f None admin] RBAC: Adding query filter params (name=member) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:19.807146 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aebed8-3ef4-4dcb-9d8d-b1410330625f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:19.808515 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aebed8-3ef4-4dcb-9d8d-b1410330625f None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:19.817373 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 199/384] 10.222.0.57 () {56 vars in 1101 bytes} [Wed Aug 10 08:23:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:19.823559 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4141c321-fefe-4c7e-b7b7-26f9d92c8dc1 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:19.933267 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4141c321-fefe-4c7e-b7b7-26f9d92c8dc1 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:19.969471 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb17ff9c-cb73-48d0-af4f-7a0a4f0fc816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:19.970161 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb17ff9c-cb73-48d0-af4f-7a0a4f0fc816 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:19.970441 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb17ff9c-cb73-48d0-af4f-7a0a4f0fc816 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:19.970712 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb17ff9c-cb73-48d0-af4f-7a0a4f0fc816 None admin] PATH_INFO: `/v3/groups/db43b5c3821f4fda98bfe4343e15a822` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:19.971868 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb17ff9c-cb73-48d0-af4f-7a0a4f0fc816 None admin] RBAC: Authorizing `identity:get_group(group_id=db43b5c3821f4fda98bfe4343e15a822)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:19.973404 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb17ff9c-cb73-48d0-af4f-7a0a4f0fc816 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:19.975536 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 186/385] 10.222.0.57 () {56 vars in 1180 bytes} [Wed Aug 10 08:23:19 2022] GET /identity/v3/groups/db43b5c3821f4fda98bfe4343e15a822 => generated 226 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:19.981159 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aebed8-3ef4-4dcb-9d8d-b1410330625f None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:20.115406 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aebed8-3ef4-4dcb-9d8d-b1410330625f None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:20.131244 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:20.154325 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b424831e-2baf-4ff9-bbea-94245138b470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:20.154962 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b424831e-2baf-4ff9-bbea-94245138b470 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:20.155248 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b424831e-2baf-4ff9-bbea-94245138b470 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:20.155522 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b424831e-2baf-4ff9-bbea-94245138b470 None admin] PATH_INFO: `/v3/projects/0f2b0269b8b24890b55cca8078954ba1` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:20.156650 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b424831e-2baf-4ff9-bbea-94245138b470 None admin] RBAC: Authorizing `identity:get_project(project_id=0f2b0269b8b24890b55cca8078954ba1)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:20.158273 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b424831e-2baf-4ff9-bbea-94245138b470 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:20.160216 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 200/386] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Aug 10 08:23:19 2022] GET /identity/v3/projects/0f2b0269b8b24890b55cca8078954ba1 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:20.166417 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb17ff9c-cb73-48d0-af4f-7a0a4f0fc816 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:20.189509 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:20.288325 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb17ff9c-cb73-48d0-af4f-7a0a4f0fc816 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:20.328795 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c103d654-27f2-459e-b15d-564ac14aa858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:20.329543 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c103d654-27f2-459e-b15d-564ac14aa858 None admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:20.329825 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c103d654-27f2-459e-b15d-564ac14aa858 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:20.330152 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c103d654-27f2-459e-b15d-564ac14aa858 None admin] PATH_INFO: `/v3/projects/0f2b0269b8b24890b55cca8078954ba1/groups/db43b5c3821f4fda98bfe4343e15a822/roles/44b64db484f24a46bf0b7ced46b519bf` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:20.338228 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c103d654-27f2-459e-b15d-564ac14aa858 None admin] RBAC: Authorizing `identity:create_grant(project_id=0f2b0269b8b24890b55cca8078954ba1, group_id=db43b5c3821f4fda98bfe4343e15a822, role_id=44b64db484f24a46bf0b7ced46b519bf)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:20.339732 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c103d654-27f2-459e-b15d-564ac14aa858 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:20.365816 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c49c5b-69b6-4749-89f9-4b9d3c1e49bd None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:20.365996 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c49c5b-69b6-4749-89f9-4b9d3c1e49bd None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:20.366332 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c49c5b-69b6-4749-89f9-4b9d3c1e49bd None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:20.367678 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 201/387] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:23:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 08:23:20.372898 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50ee641-26e8-430e-8152-e35a5c41dde0 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:20.373204 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50ee641-26e8-430e-8152-e35a5c41dde0 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:20.373480 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50ee641-26e8-430e-8152-e35a5c41dde0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:20.403661 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-f50ee641-26e8-430e-8152-e35a5c41dde0 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:23:20.404938 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-f50ee641-26e8-430e-8152-e35a5c41dde0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:23:20.413668 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 187/388] 10.222.0.57 () {58 vars in 1442 bytes} [Wed Aug 10 08:23:20 2022] PUT /identity/v3/projects/0f2b0269b8b24890b55cca8078954ba1/groups/db43b5c3821f4fda98bfe4343e15a822/roles/44b64db484f24a46bf0b7ced46b519bf => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 08:23:20.494056 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 202/389] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:23:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 08:23:20.518282 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c103d654-27f2-459e-b15d-564ac14aa858 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:20.620785 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:20.645206 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c103d654-27f2-459e-b15d-564ac14aa858 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:20.690686 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba94de3-23d5-4155-9832-a8e8f6fc4e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:20.691321 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba94de3-23d5-4155-9832-a8e8f6fc4e74 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:20.691597 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba94de3-23d5-4155-9832-a8e8f6fc4e74 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:20.691863 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba94de3-23d5-4155-9832-a8e8f6fc4e74 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:20.702298 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba94de3-23d5-4155-9832-a8e8f6fc4e74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 08:23:20.707806 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-aba94de3-23d5-4155-9832-a8e8f6fc4e74 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 08:23:20.707806 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:20.707806 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:20.707806 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:20.707806 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:20.707806 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:20.707806 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:20.707806 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:20.707806 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:20.707806 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:20.707806 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:20.707806 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:20.707806 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 08:23:20.707806 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 08:23:20.707806 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 08:23:20.707806 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 08:23:20.708993 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 08:23:20.708993 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 08:23:20.708993 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 08:23:20.708993 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 08:23:20.708993 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:20.708993 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:20.708993 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:20.708993 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:20.708993 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:20.708993 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:20.708993 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:20.708993 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:20.708993 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:20.708993 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:20.708993 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:20.708993 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:20.709862 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:20.709862 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:20.709862 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:20.709862 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:20.709862 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 08:23:20.709862 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 08:23:20.709862 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 08:23:20.709862 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 08:23:20.709862 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 08:23:20.709862 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 08:23:20.709862 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 08:23:20.709862 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:23:20.709862 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 188/390] 10.222.0.57 () {56 vars in 1114 bytes} [Wed Aug 10 08:23:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:20.715679 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50ee641-26e8-430e-8152-e35a5c41dde0 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:20.828206 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50ee641-26e8-430e-8152-e35a5c41dde0 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:20.843763 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:20.877319 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d9ce0b-9611-4c70-bfd2-afa2f50a2806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:20.877860 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d9ce0b-9611-4c70-bfd2-afa2f50a2806 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:20.878162 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d9ce0b-9611-4c70-bfd2-afa2f50a2806 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:20.878410 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d9ce0b-9611-4c70-bfd2-afa2f50a2806 None admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:20.879078 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d9ce0b-9611-4c70-bfd2-afa2f50a2806 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:20.879380 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d9ce0b-9611-4c70-bfd2-afa2f50a2806 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:20.880939 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d9ce0b-9611-4c70-bfd2-afa2f50a2806 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:20.892079 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 203/391] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Aug 10 08:23:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:20.898409 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba94de3-23d5-4155-9832-a8e8f6fc4e74 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:21.021155 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba94de3-23d5-4155-9832-a8e8f6fc4e74 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:21.065797 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a65c69e-64da-4a57-ae4e-8dfa1dedf6b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:21.066493 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a65c69e-64da-4a57-ae4e-8dfa1dedf6b2 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:21.066781 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a65c69e-64da-4a57-ae4e-8dfa1dedf6b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:21.067050 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a65c69e-64da-4a57-ae4e-8dfa1dedf6b2 None admin] PATH_INFO: `/v3/groups/db43b5c3821f4fda98bfe4343e15a822` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:21.073628 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a65c69e-64da-4a57-ae4e-8dfa1dedf6b2 None admin] RBAC: Authorizing `identity:get_group(group_id=db43b5c3821f4fda98bfe4343e15a822)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:21.075097 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a65c69e-64da-4a57-ae4e-8dfa1dedf6b2 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:21.076976 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 189/392] 10.222.0.57 () {56 vars in 1180 bytes} [Wed Aug 10 08:23:20 2022] GET /identity/v3/groups/db43b5c3821f4fda98bfe4343e15a822 => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:21.082470 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d9ce0b-9611-4c70-bfd2-afa2f50a2806 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:21.207291 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d9ce0b-9611-4c70-bfd2-afa2f50a2806 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:21.250033 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f334d73-13de-43ac-9a4f-e836ba2586a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:21.250673 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f334d73-13de-43ac-9a4f-e836ba2586a9 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:21.250952 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f334d73-13de-43ac-9a4f-e836ba2586a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:21.251225 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f334d73-13de-43ac-9a4f-e836ba2586a9 None admin] PATH_INFO: `/v3/projects/0f2b0269b8b24890b55cca8078954ba1` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:21.252377 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f334d73-13de-43ac-9a4f-e836ba2586a9 None admin] RBAC: Authorizing `identity:get_project(project_id=0f2b0269b8b24890b55cca8078954ba1)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:21.253846 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f334d73-13de-43ac-9a4f-e836ba2586a9 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:21.256247 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 204/393] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Aug 10 08:23:21 2022] GET /identity/v3/projects/0f2b0269b8b24890b55cca8078954ba1 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:21.262126 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a65c69e-64da-4a57-ae4e-8dfa1dedf6b2 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:21.271895 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:21.382068 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a65c69e-64da-4a57-ae4e-8dfa1dedf6b2 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:21.435647 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75facb82-54fd-4493-bb3d-3cef847af9aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:21.436324 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75facb82-54fd-4493-bb3d-3cef847af9aa None admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:21.436606 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75facb82-54fd-4493-bb3d-3cef847af9aa None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:21.436878 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75facb82-54fd-4493-bb3d-3cef847af9aa None admin] PATH_INFO: `/v3/projects/0f2b0269b8b24890b55cca8078954ba1/groups/db43b5c3821f4fda98bfe4343e15a822/roles/ccfc8d3fdf2d458299df29001278930f` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:21.439106 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75facb82-54fd-4493-bb3d-3cef847af9aa None admin] RBAC: Authorizing `identity:create_grant(project_id=0f2b0269b8b24890b55cca8078954ba1, group_id=db43b5c3821f4fda98bfe4343e15a822, role_id=ccfc8d3fdf2d458299df29001278930f)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:21.440545 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75facb82-54fd-4493-bb3d-3cef847af9aa None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:21.534328 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 190/394] 10.222.0.57 () {58 vars in 1442 bytes} [Wed Aug 10 08:23:21 2022] PUT /identity/v3/projects/0f2b0269b8b24890b55cca8078954ba1/groups/db43b5c3821f4fda98bfe4343e15a822/roles/ccfc8d3fdf2d458299df29001278930f => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:23:21.543211 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb82bdc-2353-4fad-8797-9801ccd41c8c None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:21.543496 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb82bdc-2353-4fad-8797-9801ccd41c8c None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:21.549458 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb82bdc-2353-4fad-8797-9801ccd41c8c None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:21.549458 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 205/395] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:23:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:23:21.554803 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c773df6-0621-4097-a051-3d05c07d1c75 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:21.555085 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c773df6-0621-4097-a051-3d05c07d1c75 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:21.555364 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c773df6-0621-4097-a051-3d05c07d1c75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:21.581418 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-6c773df6-0621-4097-a051-3d05c07d1c75 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:23:21.589439 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-6c773df6-0621-4097-a051-3d05c07d1c75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:23:21.665971 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:21.678472 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 191/396] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:23:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 08:23:21.719468 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb82bdc-2353-4fad-8797-9801ccd41c8c None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:21.772107 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:21.844206 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb82bdc-2353-4fad-8797-9801ccd41c8c None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:21.889737 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0eb0cc7-0c08-40f1-8158-0b8773eb021e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:21.890615 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eb0cc7-0c08-40f1-8158-0b8773eb021e None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:21.890909 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eb0cc7-0c08-40f1-8158-0b8773eb021e None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:21.891239 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eb0cc7-0c08-40f1-8158-0b8773eb021e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:21.908159 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0eb0cc7-0c08-40f1-8158-0b8773eb021e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 08:23:21.915068 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-c0eb0cc7-0c08-40f1-8158-0b8773eb021e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 08:23:21.915068 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:21.915068 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:21.915068 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:21.915068 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:21.915068 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:21.915068 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:21.915068 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:21.915068 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:21.915068 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:21.915068 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:21.915068 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:21.915068 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 08:23:21.915068 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 08:23:21.915068 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 08:23:21.915068 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 08:23:21.915990 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 08:23:21.915990 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 08:23:21.915990 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 08:23:21.915990 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 08:23:21.915990 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:21.915990 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:21.915990 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:21.915990 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:21.915990 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:21.915990 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:21.915990 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:21.915990 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:21.915990 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:21.915990 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:21.915990 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:21.915990 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:21.916816 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:21.916816 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:21.916816 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:21.916816 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:21.916816 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 08:23:21.916816 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 08:23:21.916816 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 08:23:21.916816 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 08:23:21.916816 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 08:23:21.916816 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 08:23:21.916816 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 08:23:21.916816 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:23:21.916816 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 206/397] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Aug 10 08:23:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:21.922791 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c773df6-0621-4097-a051-3d05c07d1c75 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:21.971327 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365df4db-2a30-41d9-84c2-fdb409df3ca6 None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:21.971965 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365df4db-2a30-41d9-84c2-fdb409df3ca6 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:21.972440 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365df4db-2a30-41d9-84c2-fdb409df3ca6 None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:21.975223 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 207/398] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:23:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:23:21.980956 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb8bd10-d8d3-4d32-a0bd-ffd1c7e0b4e7 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:21.981226 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb8bd10-d8d3-4d32-a0bd-ffd1c7e0b4e7 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:21.981498 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb8bd10-d8d3-4d32-a0bd-ffd1c7e0b4e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:22.013943 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-1eb8bd10-d8d3-4d32-a0bd-ffd1c7e0b4e7 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:23:22.015156 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-1eb8bd10-d8d3-4d32-a0bd-ffd1c7e0b4e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:23:22.040333 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c773df6-0621-4097-a051-3d05c07d1c75 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:22.076290 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce827de5-d83a-48ba-8ffc-ce3c6f3242d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:22.076907 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce827de5-d83a-48ba-8ffc-ce3c6f3242d6 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:22.077210 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce827de5-d83a-48ba-8ffc-ce3c6f3242d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:22.077478 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce827de5-d83a-48ba-8ffc-ce3c6f3242d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:22.078213 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce827de5-d83a-48ba-8ffc-ce3c6f3242d6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:22.078454 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce827de5-d83a-48ba-8ffc-ce3c6f3242d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:22.079821 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce827de5-d83a-48ba-8ffc-ce3c6f3242d6 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:22.088933 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 192/399] 10.222.0.57 () {56 vars in 1101 bytes} [Wed Aug 10 08:23:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:22.100552 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce827de5-d83a-48ba-8ffc-ce3c6f3242d6 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:22.147167 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 208/400] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:23:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 08:23:22.183454 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb8bd10-d8d3-4d32-a0bd-ffd1c7e0b4e7 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:22.196062 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:22.234449 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:22.261395 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce827de5-d83a-48ba-8ffc-ce3c6f3242d6 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:22.298673 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110b55bd-8189-4667-be9f-2ef54263e165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:22.299623 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110b55bd-8189-4667-be9f-2ef54263e165 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:22.300070 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110b55bd-8189-4667-be9f-2ef54263e165 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:22.300544 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110b55bd-8189-4667-be9f-2ef54263e165 None admin] PATH_INFO: `/v3/groups/db43b5c3821f4fda98bfe4343e15a822` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:22.301908 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110b55bd-8189-4667-be9f-2ef54263e165 None admin] RBAC: Authorizing `identity:get_group(group_id=db43b5c3821f4fda98bfe4343e15a822)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:22.303488 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110b55bd-8189-4667-be9f-2ef54263e165 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:22.305573 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 193/401] 10.222.0.57 () {56 vars in 1180 bytes} [Wed Aug 10 08:23:22 2022] GET /identity/v3/groups/db43b5c3821f4fda98bfe4343e15a822 => generated 226 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:22.306969 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42280716-d137-4e3c-bc75-73f8ce4175d0 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:22.307397 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42280716-d137-4e3c-bc75-73f8ce4175d0 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:22.307809 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42280716-d137-4e3c-bc75-73f8ce4175d0 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:22.309246 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 194/402] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:23:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:23:22.313786 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42280716-d137-4e3c-bc75-73f8ce4175d0 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:22.323472 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb8bd10-d8d3-4d32-a0bd-ffd1c7e0b4e7 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:22.376300 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b504fdb-8e02-4ced-95ba-83c527637b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:22.377071 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b504fdb-8e02-4ced-95ba-83c527637b89 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:22.377245 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b504fdb-8e02-4ced-95ba-83c527637b89 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:22.377513 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b504fdb-8e02-4ced-95ba-83c527637b89 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:22.410211 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b504fdb-8e02-4ced-95ba-83c527637b89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 08:23:22.416636 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-3b504fdb-8e02-4ced-95ba-83c527637b89 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 08:23:22.416636 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:22.416636 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:22.416636 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:22.416636 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:22.416636 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:22.416636 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:22.416636 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:22.416636 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:22.416636 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:22.416636 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:22.416636 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:22.416636 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 08:23:22.416636 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 08:23:22.416636 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 08:23:22.416636 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 08:23:22.421427 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 08:23:22.421427 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 08:23:22.421427 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 08:23:22.421427 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 08:23:22.421427 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:22.421427 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:22.421427 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:22.421427 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:22.421427 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:22.421427 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:22.421427 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:22.421427 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:22.421427 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:22.421427 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:22.421427 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:22.421427 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:22.422231 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:22.422231 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:22.422231 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:22.422231 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:22.422231 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 08:23:22.422231 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 08:23:22.422231 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 08:23:22.422231 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 08:23:22.422231 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 08:23:22.422231 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 08:23:22.422231 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 08:23:22.422231 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:23:22.422231 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 209/403] 10.222.0.57 () {56 vars in 1114 bytes} [Wed Aug 10 08:23:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:22.422231 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d449272b-cc57-4190-bcb7-c37ed66e49bc None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:22.422991 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d449272b-cc57-4190-bcb7-c37ed66e49bc None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:22.422991 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d449272b-cc57-4190-bcb7-c37ed66e49bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:22.443207 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-d449272b-cc57-4190-bcb7-c37ed66e49bc None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:23:22.445212 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-d449272b-cc57-4190-bcb7-c37ed66e49bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:23:22.467962 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42280716-d137-4e3c-bc75-73f8ce4175d0 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:22.507838 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:22.519546 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06162d7-dc7c-48cc-8346-312b138f9783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:22.521185 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06162d7-dc7c-48cc-8346-312b138f9783 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:22.522309 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06162d7-dc7c-48cc-8346-312b138f9783 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:22.522785 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06162d7-dc7c-48cc-8346-312b138f9783 None admin] PATH_INFO: `/v3/projects/201dfacf93654f63a4732891e33d5039` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:22.524489 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06162d7-dc7c-48cc-8346-312b138f9783 None admin] RBAC: Authorizing `identity:get_project(project_id=201dfacf93654f63a4732891e33d5039)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:22.527121 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06162d7-dc7c-48cc-8346-312b138f9783 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:22.529689 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 195/404] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Aug 10 08:23:22 2022] GET /identity/v3/projects/201dfacf93654f63a4732891e33d5039 => generated 302 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:22.531239 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06162d7-dc7c-48cc-8346-312b138f9783 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:22.542982 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 210/405] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:23:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 08:23:22.544229 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d449272b-cc57-4190-bcb7-c37ed66e49bc None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:22.576645 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:22.651676 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06162d7-dc7c-48cc-8346-312b138f9783 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:22.673114 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d449272b-cc57-4190-bcb7-c37ed66e49bc None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:22.688503 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1215625c-4444-4836-9fa4-2220dfe1d02f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:22.689222 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1215625c-4444-4836-9fa4-2220dfe1d02f None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:22.689695 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1215625c-4444-4836-9fa4-2220dfe1d02f None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:22.690157 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1215625c-4444-4836-9fa4-2220dfe1d02f None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:22.691446 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1215625c-4444-4836-9fa4-2220dfe1d02f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:22.692150 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1215625c-4444-4836-9fa4-2220dfe1d02f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:22.693985 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1215625c-4444-4836-9fa4-2220dfe1d02f None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:22.706204 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 196/406] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Aug 10 08:23:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:22.707137 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1215625c-4444-4836-9fa4-2220dfe1d02f None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:22.711627 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44af800-6f79-4885-81cb-c6f67d67cb93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:22.712598 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44af800-6f79-4885-81cb-c6f67d67cb93 None admin] REQUEST_METHOD: `PUT` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:22.713129 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44af800-6f79-4885-81cb-c6f67d67cb93 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:22.713634 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44af800-6f79-4885-81cb-c6f67d67cb93 None admin] PATH_INFO: `/v3/projects/201dfacf93654f63a4732891e33d5039/groups/db43b5c3821f4fda98bfe4343e15a822/roles/44b64db484f24a46bf0b7ced46b519bf` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:22.715851 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44af800-6f79-4885-81cb-c6f67d67cb93 None admin] RBAC: Authorizing `identity:create_grant(project_id=201dfacf93654f63a4732891e33d5039, group_id=db43b5c3821f4fda98bfe4343e15a822, role_id=44b64db484f24a46bf0b7ced46b519bf)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:22.717630 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44af800-6f79-4885-81cb-c6f67d67cb93 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:22.774776 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:22.820296 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1215625c-4444-4836-9fa4-2220dfe1d02f None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:22.855312 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bf1acf-1541-4793-8f3d-d7df0f02f7a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:22.856322 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bf1acf-1541-4793-8f3d-d7df0f02f7a4 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:22.856805 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bf1acf-1541-4793-8f3d-d7df0f02f7a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:22.857334 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bf1acf-1541-4793-8f3d-d7df0f02f7a4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:22.869307 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bf1acf-1541-4793-8f3d-d7df0f02f7a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 08:23:22.873741 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 211/407] 10.222.0.57 () {58 vars in 1442 bytes} [Wed Aug 10 08:23:22 2022] PUT /identity/v3/projects/201dfacf93654f63a4732891e33d5039/groups/db43b5c3821f4fda98bfe4343e15a822/roles/44b64db484f24a46bf0b7ced46b519bf => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:23:22.874976 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44af800-6f79-4885-81cb-c6f67d67cb93 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:22.882753 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-d9bf1acf-1541-4793-8f3d-d7df0f02f7a4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 08:23:22.882753 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:22.882753 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:22.882753 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:22.882753 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:22.882753 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:22.882753 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:22.882753 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:22.882753 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:22.882753 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:22.882753 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:22.882753 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:22.882753 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 08:23:22.882753 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 08:23:22.882753 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 08:23:22.882753 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 08:23:22.883677 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 08:23:22.883677 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 08:23:22.883677 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 08:23:22.883677 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 08:23:22.883677 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:22.883677 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:22.883677 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:22.883677 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:22.883677 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:22.883677 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:22.883677 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:22.883677 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:22.883677 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:22.883677 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:22.883677 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:22.883677 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:22.884508 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:22.884508 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:22.884508 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:22.884508 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:22.884508 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 08:23:22.884508 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 08:23:22.884508 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 08:23:22.884508 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 08:23:22.884508 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 08:23:22.884508 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 08:23:22.884508 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 08:23:22.884508 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:23:22.884508 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 197/408] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Aug 10 08:23:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:22.888159 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bf1acf-1541-4793-8f3d-d7df0f02f7a4 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:22.993785 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:23.017517 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44af800-6f79-4885-81cb-c6f67d67cb93 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:23.060065 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb9b775-1c66-4229-9e5f-05571b3b56a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:23.061110 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb9b775-1c66-4229-9e5f-05571b3b56a0 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:23.061612 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb9b775-1c66-4229-9e5f-05571b3b56a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:23.062153 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb9b775-1c66-4229-9e5f-05571b3b56a0 None admin] PATH_INFO: `/v3/groups/db43b5c3821f4fda98bfe4343e15a822` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:23.064568 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb9b775-1c66-4229-9e5f-05571b3b56a0 None admin] RBAC: Authorizing `identity:get_group(group_id=db43b5c3821f4fda98bfe4343e15a822)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:23.066879 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb9b775-1c66-4229-9e5f-05571b3b56a0 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:23.068852 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 212/409] 10.222.0.57 () {56 vars in 1180 bytes} [Wed Aug 10 08:23:22 2022] GET /identity/v3/groups/db43b5c3821f4fda98bfe4343e15a822 => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:23.074790 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb9b775-1c66-4229-9e5f-05571b3b56a0 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:23.099415 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bf1acf-1541-4793-8f3d-d7df0f02f7a4 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:23.120274 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:23.141169 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474b0936-238b-48be-9ca0-b18b880810c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:23.141866 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474b0936-238b-48be-9ca0-b18b880810c0 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:23.142143 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474b0936-238b-48be-9ca0-b18b880810c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:23.142411 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474b0936-238b-48be-9ca0-b18b880810c0 None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:23.143218 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474b0936-238b-48be-9ca0-b18b880810c0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:23.143523 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474b0936-238b-48be-9ca0-b18b880810c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:23.146858 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474b0936-238b-48be-9ca0-b18b880810c0 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:23.158960 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 198/410] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Aug 10 08:23:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:23.182087 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474b0936-238b-48be-9ca0-b18b880810c0 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:23.218349 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb9b775-1c66-4229-9e5f-05571b3b56a0 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:23.265337 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2eb6a4-fd21-44ab-af45-6467174561b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:23.265337 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2eb6a4-fd21-44ab-af45-6467174561b6 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:23.265337 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2eb6a4-fd21-44ab-af45-6467174561b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:23.265337 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2eb6a4-fd21-44ab-af45-6467174561b6 None admin] PATH_INFO: `/v3/projects/201dfacf93654f63a4732891e33d5039` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:23.265729 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2eb6a4-fd21-44ab-af45-6467174561b6 None admin] RBAC: Authorizing `identity:get_project(project_id=201dfacf93654f63a4732891e33d5039)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:23.265729 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2eb6a4-fd21-44ab-af45-6467174561b6 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:23.266448 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 213/411] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Aug 10 08:23:23 2022] GET /identity/v3/projects/201dfacf93654f63a4732891e33d5039 => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:23.276669 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2eb6a4-fd21-44ab-af45-6467174561b6 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:23.322900 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:23.391255 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474b0936-238b-48be-9ca0-b18b880810c0 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:23.444431 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3990af1f-4101-43c2-beff-93f7269a7325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:23.445098 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3990af1f-4101-43c2-beff-93f7269a7325 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:23.445380 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3990af1f-4101-43c2-beff-93f7269a7325 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:23.445654 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3990af1f-4101-43c2-beff-93f7269a7325 None admin] PATH_INFO: `/v3/groups/a38ec50e49fb4028871d1d6f28421e35` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:23.446837 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3990af1f-4101-43c2-beff-93f7269a7325 None admin] RBAC: Authorizing `identity:get_group(group_id=a38ec50e49fb4028871d1d6f28421e35)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:23.448329 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3990af1f-4101-43c2-beff-93f7269a7325 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:23.452193 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 199/412] 10.222.0.57 () {56 vars in 1180 bytes} [Wed Aug 10 08:23:23 2022] GET /identity/v3/groups/a38ec50e49fb4028871d1d6f28421e35 => generated 229 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:23.461927 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3990af1f-4101-43c2-beff-93f7269a7325 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:23.482269 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2eb6a4-fd21-44ab-af45-6467174561b6 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:23.511210 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:23.530307 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6ac4b8-dc39-43ea-a07b-b00d7ed1f01a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:23.530882 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6ac4b8-dc39-43ea-a07b-b00d7ed1f01a None admin] REQUEST_METHOD: `PUT` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:23.531157 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6ac4b8-dc39-43ea-a07b-b00d7ed1f01a None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:23.531424 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6ac4b8-dc39-43ea-a07b-b00d7ed1f01a None admin] PATH_INFO: `/v3/projects/201dfacf93654f63a4732891e33d5039/groups/db43b5c3821f4fda98bfe4343e15a822/roles/ccfc8d3fdf2d458299df29001278930f` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:23.535958 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6ac4b8-dc39-43ea-a07b-b00d7ed1f01a None admin] RBAC: Authorizing `identity:create_grant(project_id=201dfacf93654f63a4732891e33d5039, group_id=db43b5c3821f4fda98bfe4343e15a822, role_id=ccfc8d3fdf2d458299df29001278930f)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:23.538913 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6ac4b8-dc39-43ea-a07b-b00d7ed1f01a None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:23.570418 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3990af1f-4101-43c2-beff-93f7269a7325 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:23.600943 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 214/413] 10.222.0.57 () {58 vars in 1442 bytes} [Wed Aug 10 08:23:23 2022] PUT /identity/v3/projects/201dfacf93654f63a4732891e33d5039/groups/db43b5c3821f4fda98bfe4343e15a822/roles/ccfc8d3fdf2d458299df29001278930f => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:23:23.603862 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33df9a5-8cf2-48b5-8f3e-7ceac842ff35 None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:23.606104 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33df9a5-8cf2-48b5-8f3e-7ceac842ff35 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:23.606398 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33df9a5-8cf2-48b5-8f3e-7ceac842ff35 None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:23.608435 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 215/414] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:23:23 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:23:23.615998 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb0f439-5868-48b5-830b-94f9a0811a58 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:23.616335 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb0f439-5868-48b5-830b-94f9a0811a58 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:23.616609 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb0f439-5868-48b5-830b-94f9a0811a58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:23.623047 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92884167-fc05-46a5-8d9d-8873d93b12de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:23.624014 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92884167-fc05-46a5-8d9d-8873d93b12de None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:23.624480 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92884167-fc05-46a5-8d9d-8873d93b12de None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:23.625007 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92884167-fc05-46a5-8d9d-8873d93b12de None admin] PATH_INFO: `/v3/projects/fe95c65d461a414fa25f77dc13eab47b` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:23.626607 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92884167-fc05-46a5-8d9d-8873d93b12de None admin] RBAC: Authorizing `identity:get_project(project_id=fe95c65d461a414fa25f77dc13eab47b)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:23.628296 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92884167-fc05-46a5-8d9d-8873d93b12de None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:23.630792 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 200/415] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Aug 10 08:23:23 2022] GET /identity/v3/projects/fe95c65d461a414fa25f77dc13eab47b => generated 344 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:23.646367 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92884167-fc05-46a5-8d9d-8873d93b12de None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:23.652885 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-6cb0f439-5868-48b5-830b-94f9a0811a58 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:23:23.656229 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-6cb0f439-5868-48b5-830b-94f9a0811a58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:23:23.784851 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 216/416] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:23:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 08:23:23.810555 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb0f439-5868-48b5-830b-94f9a0811a58 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:23.822964 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:23.829160 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92884167-fc05-46a5-8d9d-8873d93b12de None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:23.857294 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:23.875021 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71159936-c5dc-42dd-a47a-a0655d306cca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:23.875815 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71159936-c5dc-42dd-a47a-a0655d306cca None admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:23.876231 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71159936-c5dc-42dd-a47a-a0655d306cca None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:23.876594 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71159936-c5dc-42dd-a47a-a0655d306cca None admin] PATH_INFO: `/v3/projects/fe95c65d461a414fa25f77dc13eab47b/groups/a38ec50e49fb4028871d1d6f28421e35/roles/ec3751692d9c4ecf81b4a71c936c46be` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:23.879244 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71159936-c5dc-42dd-a47a-a0655d306cca None admin] RBAC: Authorizing `identity:create_grant(project_id=fe95c65d461a414fa25f77dc13eab47b, group_id=a38ec50e49fb4028871d1d6f28421e35, role_id=ec3751692d9c4ecf81b4a71c936c46be)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:23.882753 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71159936-c5dc-42dd-a47a-a0655d306cca None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:23.931024 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb0f439-5868-48b5-830b-94f9a0811a58 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:23.955065 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 201/417] 10.222.0.57 () {58 vars in 1442 bytes} [Wed Aug 10 08:23:23 2022] PUT /identity/v3/projects/fe95c65d461a414fa25f77dc13eab47b/groups/a38ec50e49fb4028871d1d6f28421e35/roles/ec3751692d9c4ecf81b4a71c936c46be => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:23:23.969519 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99b40cd-7f7e-407d-b1f9-7e5656c2d445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:23.970453 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99b40cd-7f7e-407d-b1f9-7e5656c2d445 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:23.970892 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99b40cd-7f7e-407d-b1f9-7e5656c2d445 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:23.971308 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99b40cd-7f7e-407d-b1f9-7e5656c2d445 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:23.984261 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99b40cd-7f7e-407d-b1f9-7e5656c2d445 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 08:23:23.989508 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-b99b40cd-7f7e-407d-b1f9-7e5656c2d445 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 08:23:23.989508 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:23.989508 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:23.989508 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:23.989508 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:23.989508 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:23.989508 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:23.989508 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:23.989508 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:23.989508 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:23.989508 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:23.989508 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:23.989508 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 08:23:23.989508 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 08:23:23.989508 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 08:23:23.989508 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 08:23:23.990594 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 08:23:23.990594 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 08:23:23.990594 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 08:23:23.990594 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 08:23:23.990594 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:23.990594 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:23.990594 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:23.990594 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:23.990594 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:23.990594 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:23.990594 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:23.990594 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:23.990594 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:23.990594 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:23.990594 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:23.990594 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:23.991572 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:23.991572 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:23.991572 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:23.991572 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:23.991572 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 08:23:23.991572 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 08:23:23.991572 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 08:23:23.991572 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 08:23:23.991572 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 08:23:23.991572 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 08:23:23.991572 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 08:23:23.991572 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:23:23.998821 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 217/418] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Aug 10 08:23:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 08:23:24.000401 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71159936-c5dc-42dd-a47a-a0655d306cca None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:24.128287 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71159936-c5dc-42dd-a47a-a0655d306cca None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:24.174802 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a349ed-f401-4c98-8085-d2a72f48af61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:24.175375 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9c773a-76af-4733-a94f-d1efc469a21b None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:24.175496 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a349ed-f401-4c98-8085-d2a72f48af61 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:24.175706 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a349ed-f401-4c98-8085-d2a72f48af61 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:24.175940 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a349ed-f401-4c98-8085-d2a72f48af61 None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:24.176386 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9c773a-76af-4733-a94f-d1efc469a21b None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:24.176678 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a349ed-f401-4c98-8085-d2a72f48af61 None admin] RBAC: Adding query filter params (name=member) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:24.177605 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a349ed-f401-4c98-8085-d2a72f48af61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:24.178052 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9c773a-76af-4733-a94f-d1efc469a21b None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:24.178975 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a349ed-f401-4c98-8085-d2a72f48af61 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:24.179877 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 218/419] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:23:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:23:24.186258 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d794cb8-f121-4d88-a670-65d907724970 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:24.186796 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d794cb8-f121-4d88-a670-65d907724970 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:24.187266 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d794cb8-f121-4d88-a670-65d907724970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:24.188060 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 202/420] 10.222.0.57 () {56 vars in 1101 bytes} [Wed Aug 10 08:23:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:24.197254 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a349ed-f401-4c98-8085-d2a72f48af61 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:24.210016 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-9d794cb8-f121-4d88-a670-65d907724970 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:23:24.212530 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-9d794cb8-f121-4d88-a670-65d907724970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:23:24.243170 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:24.283981 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:24.293371 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 219/421] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:23:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 08:23:24.308343 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a349ed-f401-4c98-8085-d2a72f48af61 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:24.318844 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d794cb8-f121-4d88-a670-65d907724970 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:24.366907 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c56608-853a-46ed-873f-ad711a8dca07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:24.367390 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c56608-853a-46ed-873f-ad711a8dca07 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:24.367667 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c56608-853a-46ed-873f-ad711a8dca07 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:24.367933 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c56608-853a-46ed-873f-ad711a8dca07 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:24.445087 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d794cb8-f121-4d88-a670-65d907724970 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:24.484832 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbf3959-833b-43f2-8a32-adc72b5a8480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:24.485894 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbf3959-833b-43f2-8a32-adc72b5a8480 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:24.486380 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbf3959-833b-43f2-8a32-adc72b5a8480 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:24.486872 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbf3959-833b-43f2-8a32-adc72b5a8480 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:24.499965 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbf3959-833b-43f2-8a32-adc72b5a8480 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 08:23:24.507441 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-6fbf3959-833b-43f2-8a32-adc72b5a8480 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 08:23:24.507441 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:24.507441 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:24.507441 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:24.507441 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:24.507441 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:24.507441 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:24.507441 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:24.507441 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:24.507441 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:24.507441 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:24.507441 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:24.507441 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 08:23:24.507441 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 08:23:24.507441 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 08:23:24.507441 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 08:23:24.508366 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 08:23:24.508366 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 08:23:24.508366 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 08:23:24.508366 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 08:23:24.508366 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:24.508366 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:24.508366 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:24.508366 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:24.508366 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:24.508366 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:24.508366 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:24.508366 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:24.508366 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:24.508366 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:24.508366 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:24.508366 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:24.509240 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:24.509240 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:24.509240 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:24.509240 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:24.509240 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 08:23:24.509240 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 08:23:24.509240 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 08:23:24.509240 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 08:23:24.509240 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 08:23:24.509240 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 08:23:24.509240 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 08:23:24.509240 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:23:24.515567 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c56608-853a-46ed-873f-ad711a8dca07 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:24.515567 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c56608-853a-46ed-873f-ad711a8dca07 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:24.516082 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 220/422] 10.222.0.57 () {56 vars in 1114 bytes} [Wed Aug 10 08:23:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 08:23:24.518782 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbf3959-833b-43f2-8a32-adc72b5a8480 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:24.586894 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:24.596274 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:24.634342 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbf3959-833b-43f2-8a32-adc72b5a8480 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:24.673457 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7455744c-6efe-41d4-80a8-3b7171e50047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:24.674401 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7455744c-6efe-41d4-80a8-3b7171e50047 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:24.674863 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7455744c-6efe-41d4-80a8-3b7171e50047 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:24.675313 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7455744c-6efe-41d4-80a8-3b7171e50047 None admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:24.677164 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7455744c-6efe-41d4-80a8-3b7171e50047 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:24.677678 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7455744c-6efe-41d4-80a8-3b7171e50047 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:24.679336 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7455744c-6efe-41d4-80a8-3b7171e50047 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:24.688950 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 221/423] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Aug 10 08:23:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:23:24.696363 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7455744c-6efe-41d4-80a8-3b7171e50047 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:24.731463 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 203/424] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Aug 10 08:23:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 535 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 10 08:23:24.738416 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c56608-853a-46ed-873f-ad711a8dca07 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:24.878411 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7455744c-6efe-41d4-80a8-3b7171e50047 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:24.910228 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:24.925912 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c56608-853a-46ed-873f-ad711a8dca07 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:24.950961 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e62a098-17c3-4bd4-9655-2dc1d32fbf91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:24.951579 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e62a098-17c3-4bd4-9655-2dc1d32fbf91 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:24.951860 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e62a098-17c3-4bd4-9655-2dc1d32fbf91 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:24.952131 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e62a098-17c3-4bd4-9655-2dc1d32fbf91 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:24.976803 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edbe605-c8dc-48d8-b62e-24ae3121175e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:24.977706 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edbe605-c8dc-48d8-b62e-24ae3121175e None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:24.978161 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edbe605-c8dc-48d8-b62e-24ae3121175e None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:24.978569 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edbe605-c8dc-48d8-b62e-24ae3121175e None admin] PATH_INFO: `/v3/projects/0f2b0269b8b24890b55cca8078954ba1` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:24.979965 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edbe605-c8dc-48d8-b62e-24ae3121175e None admin] RBAC: Authorizing `identity:get_project(project_id=0f2b0269b8b24890b55cca8078954ba1)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:24.981685 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edbe605-c8dc-48d8-b62e-24ae3121175e None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:24.983648 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 204/425] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Aug 10 08:23:24 2022] GET /identity/v3/projects/0f2b0269b8b24890b55cca8078954ba1 => generated 298 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:24.990418 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edbe605-c8dc-48d8-b62e-24ae3121175e None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:24.996536 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:25.038304 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e62a098-17c3-4bd4-9655-2dc1d32fbf91 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:25.039769 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e62a098-17c3-4bd4-9655-2dc1d32fbf91 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:25.183727 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edbe605-c8dc-48d8-b62e-24ae3121175e None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:25.195153 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 222/426] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Aug 10 08:23:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 10 08:23:25.201516 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e62a098-17c3-4bd4-9655-2dc1d32fbf91 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:25.225315 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:25.241922 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41127146-9faf-451e-b8cb-75a6b20eaf0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:25.242790 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41127146-9faf-451e-b8cb-75a6b20eaf0b None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:25.243228 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41127146-9faf-451e-b8cb-75a6b20eaf0b None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:25.243500 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41127146-9faf-451e-b8cb-75a6b20eaf0b None admin] PATH_INFO: `/v3/groups/db43b5c3821f4fda98bfe4343e15a822` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:25.244626 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41127146-9faf-451e-b8cb-75a6b20eaf0b None admin] RBAC: Authorizing `identity:get_group(group_id=db43b5c3821f4fda98bfe4343e15a822)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:25.246364 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41127146-9faf-451e-b8cb-75a6b20eaf0b None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:25.248179 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 205/427] 10.222.0.57 () {56 vars in 1180 bytes} [Wed Aug 10 08:23:24 2022] GET /identity/v3/groups/db43b5c3821f4fda98bfe4343e15a822 => generated 226 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:25.249155 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08acb41-0d77-4af8-aea3-ea9fa2e0c61e None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:25.249432 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08acb41-0d77-4af8-aea3-ea9fa2e0c61e None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:25.249701 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08acb41-0d77-4af8-aea3-ea9fa2e0c61e None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:25.251139 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 206/428] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:23:25 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:23:25.253511 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08acb41-0d77-4af8-aea3-ea9fa2e0c61e None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:25.318759 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e62a098-17c3-4bd4-9655-2dc1d32fbf91 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:25.345922 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:25.357639 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f2e7d2-2431-40ce-86d9-bc2c5d77b118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:25.358391 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f2e7d2-2431-40ce-86d9-bc2c5d77b118 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:25.358730 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f2e7d2-2431-40ce-86d9-bc2c5d77b118 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:25.359059 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f2e7d2-2431-40ce-86d9-bc2c5d77b118 None admin] PATH_INFO: `/v3/projects/0f2b0269b8b24890b55cca8078954ba1` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:25.360515 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f2e7d2-2431-40ce-86d9-bc2c5d77b118 None admin] RBAC: Authorizing `identity:get_project(project_id=0f2b0269b8b24890b55cca8078954ba1)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:25.362178 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f2e7d2-2431-40ce-86d9-bc2c5d77b118 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:25.364882 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 223/429] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Aug 10 08:23:25 2022] GET /identity/v3/projects/0f2b0269b8b24890b55cca8078954ba1 => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:25.366210 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c046cda-1dab-4098-8836-55f5cceace93 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:25.366590 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c046cda-1dab-4098-8836-55f5cceace93 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:25.366961 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c046cda-1dab-4098-8836-55f5cceace93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:25.384973 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08acb41-0d77-4af8-aea3-ea9fa2e0c61e None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:25.397281 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-1c046cda-1dab-4098-8836-55f5cceace93 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:23:25.398781 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-1c046cda-1dab-4098-8836-55f5cceace93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:23:25.446797 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815b451a-e574-4a9c-8cbf-27fc834c5181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:25.447789 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815b451a-e574-4a9c-8cbf-27fc834c5181 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:25.448287 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815b451a-e574-4a9c-8cbf-27fc834c5181 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:25.448825 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815b451a-e574-4a9c-8cbf-27fc834c5181 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:25.450348 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815b451a-e574-4a9c-8cbf-27fc834c5181 None admin] RBAC: Adding query filter params (group.id=db43b5c3821f4fda98bfe4343e15a822, role.id=44b64db484f24a46bf0b7ced46b519bf, scope.project.id=0f2b0269b8b24890b55cca8078954ba1) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:25.450868 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815b451a-e574-4a9c-8cbf-27fc834c5181 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:25.453092 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815b451a-e574-4a9c-8cbf-27fc834c5181 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:25.461731 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 207/430] 10.222.0.57 () {56 vars in 1376 bytes} [Wed Aug 10 08:23:25 2022] GET /identity/v3/role_assignments?group.id=db43b5c3821f4fda98bfe4343e15a822&scope.project.id=0f2b0269b8b24890b55cca8078954ba1&role.id=44b64db484f24a46bf0b7ced46b519bf => generated 611 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:23:25.462966 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815b451a-e574-4a9c-8cbf-27fc834c5181 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:25.508944 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 224/431] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:23:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 08:23:25.557303 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c046cda-1dab-4098-8836-55f5cceace93 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:25.593394 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:25.643801 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815b451a-e574-4a9c-8cbf-27fc834c5181 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:25.688686 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1e8373-4f6d-4379-aede-6da880a0c571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:25.689707 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1e8373-4f6d-4379-aede-6da880a0c571 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:25.690194 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1e8373-4f6d-4379-aede-6da880a0c571 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:25.690740 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1e8373-4f6d-4379-aede-6da880a0c571 None admin] PATH_INFO: `/v3/groups/db43b5c3821f4fda98bfe4343e15a822` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:25.692766 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1e8373-4f6d-4379-aede-6da880a0c571 None admin] RBAC: Authorizing `identity:get_group(group_id=db43b5c3821f4fda98bfe4343e15a822)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:25.694510 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1e8373-4f6d-4379-aede-6da880a0c571 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:25.701650 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 208/432] 10.222.0.57 () {56 vars in 1180 bytes} [Wed Aug 10 08:23:25 2022] GET /identity/v3/groups/db43b5c3821f4fda98bfe4343e15a822 => generated 226 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:25.705937 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1e8373-4f6d-4379-aede-6da880a0c571 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:25.711620 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:25.743155 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c046cda-1dab-4098-8836-55f5cceace93 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:25.788734 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7af3f8-d081-4084-8a04-2505d9cfa709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:25.789402 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7af3f8-d081-4084-8a04-2505d9cfa709 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:25.789681 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7af3f8-d081-4084-8a04-2505d9cfa709 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:25.789949 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7af3f8-d081-4084-8a04-2505d9cfa709 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:25.810654 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7af3f8-d081-4084-8a04-2505d9cfa709 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 08:23:25.816121 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-6d7af3f8-d081-4084-8a04-2505d9cfa709 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 08:23:25.816121 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:25.816121 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:25.816121 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:25.816121 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:25.816121 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:25.816121 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:25.816121 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:25.816121 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:25.816121 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:25.816121 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:25.816121 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:25.816121 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 08:23:25.816121 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 08:23:25.816121 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 08:23:25.816121 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 08:23:25.817055 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 08:23:25.817055 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 08:23:25.817055 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 08:23:25.817055 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 08:23:25.817055 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:25.817055 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:25.817055 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:25.817055 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:25.817055 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:25.817055 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:25.817055 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:25.817055 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:25.817055 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:25.817055 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:25.817055 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:25.817055 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:25.817912 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:25.817912 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:25.817912 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:25.817912 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:25.817912 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 08:23:25.817912 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 08:23:25.817912 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 08:23:25.817912 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 08:23:25.817912 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 08:23:25.817912 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 08:23:25.817912 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 08:23:25.817912 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:23:25.824828 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 225/433] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Aug 10 08:23:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:25.827130 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7af3f8-d081-4084-8a04-2505d9cfa709 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:25.855507 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1e8373-4f6d-4379-aede-6da880a0c571 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:25.895861 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd57a57-7454-4f10-8d40-df8471a9c3d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:25.896777 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd57a57-7454-4f10-8d40-df8471a9c3d3 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:25.896889 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd57a57-7454-4f10-8d40-df8471a9c3d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:25.897131 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd57a57-7454-4f10-8d40-df8471a9c3d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:25.897874 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd57a57-7454-4f10-8d40-df8471a9c3d3 None admin] RBAC: Adding query filter params (group.id=db43b5c3821f4fda98bfe4343e15a822, role.id=ccfc8d3fdf2d458299df29001278930f, scope.project.id=0f2b0269b8b24890b55cca8078954ba1) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:25.898175 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd57a57-7454-4f10-8d40-df8471a9c3d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:25.899603 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd57a57-7454-4f10-8d40-df8471a9c3d3 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:25.912948 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 209/434] 10.222.0.57 () {56 vars in 1376 bytes} [Wed Aug 10 08:23:25 2022] GET /identity/v3/role_assignments?group.id=db43b5c3821f4fda98bfe4343e15a822&scope.project.id=0f2b0269b8b24890b55cca8078954ba1&role.id=ccfc8d3fdf2d458299df29001278930f => generated 611 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:25.923749 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:25.958387 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7af3f8-d081-4084-8a04-2505d9cfa709 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:26.002596 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7073c980-595b-48a1-baea-c4d6b2fa0fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:26.003582 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7073c980-595b-48a1-baea-c4d6b2fa0fd0 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:26.004098 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7073c980-595b-48a1-baea-c4d6b2fa0fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:26.004598 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7073c980-595b-48a1-baea-c4d6b2fa0fd0 None admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:26.006076 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7073c980-595b-48a1-baea-c4d6b2fa0fd0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:26.006656 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7073c980-595b-48a1-baea-c4d6b2fa0fd0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:26.008288 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7073c980-595b-48a1-baea-c4d6b2fa0fd0 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:26.023666 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 226/435] 10.222.0.57 () {56 vars in 1101 bytes} [Wed Aug 10 08:23:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:26.025527 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd57a57-7454-4f10-8d40-df8471a9c3d3 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:26.110342 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:26.151879 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd57a57-7454-4f10-8d40-df8471a9c3d3 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:26.190322 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a684de3-95d0-40a0-b919-f30642df7b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:26.190933 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a684de3-95d0-40a0-b919-f30642df7b91 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:26.191263 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a684de3-95d0-40a0-b919-f30642df7b91 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:26.191540 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a684de3-95d0-40a0-b919-f30642df7b91 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:26.276471 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a684de3-95d0-40a0-b919-f30642df7b91 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:26.277906 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a684de3-95d0-40a0-b919-f30642df7b91 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:26.283680 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9eb3a9-7948-4435-94bd-ad36da1763bc None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:26.283966 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9eb3a9-7948-4435-94bd-ad36da1763bc None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:26.284241 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9eb3a9-7948-4435-94bd-ad36da1763bc None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:26.285762 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 227/436] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:23:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:23:26.291365 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4ae042-1e0b-48f2-ba66-b234a0dfb026 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:26.291716 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4ae042-1e0b-48f2-ba66-b234a0dfb026 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:26.292058 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4ae042-1e0b-48f2-ba66-b234a0dfb026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:26.315650 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-db4ae042-1e0b-48f2-ba66-b234a0dfb026 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:23:26.316940 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-db4ae042-1e0b-48f2-ba66-b234a0dfb026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:23:26.360121 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:26.414232 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 228/437] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:23:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 08:23:26.425861 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 210/438] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Aug 10 08:23:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 10 08:23:26.428938 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4ae042-1e0b-48f2-ba66-b234a0dfb026 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:26.440152 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a684de3-95d0-40a0-b919-f30642df7b91 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:26.514019 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:26.551076 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4ae042-1e0b-48f2-ba66-b234a0dfb026 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:26.561303 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a684de3-95d0-40a0-b919-f30642df7b91 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:26.585199 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19d389e-6db4-4ddd-853f-b0f565036348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:26.585822 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19d389e-6db4-4ddd-853f-b0f565036348 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:26.586101 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19d389e-6db4-4ddd-853f-b0f565036348 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:26.586371 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19d389e-6db4-4ddd-853f-b0f565036348 None admin] PATH_INFO: `/v3/projects/201dfacf93654f63a4732891e33d5039` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:26.587485 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19d389e-6db4-4ddd-853f-b0f565036348 None admin] RBAC: Authorizing `identity:get_project(project_id=201dfacf93654f63a4732891e33d5039)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:26.588943 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19d389e-6db4-4ddd-853f-b0f565036348 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:26.590881 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 229/439] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Aug 10 08:23:26 2022] GET /identity/v3/projects/201dfacf93654f63a4732891e33d5039 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:26.591983 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f618b2e8-b74e-4118-9b48-2199bb480fdc None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:26.592261 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f618b2e8-b74e-4118-9b48-2199bb480fdc None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:26.592530 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f618b2e8-b74e-4118-9b48-2199bb480fdc None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:26.593731 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 230/440] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:23:26 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:23:26.595866 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f618b2e8-b74e-4118-9b48-2199bb480fdc None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:26.600451 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadaf518-824f-4437-a217-1e9458b4b584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:26.601124 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadaf518-824f-4437-a217-1e9458b4b584 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:26.601410 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadaf518-824f-4437-a217-1e9458b4b584 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:26.601682 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadaf518-824f-4437-a217-1e9458b4b584 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:26.612148 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadaf518-824f-4437-a217-1e9458b4b584 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 08:23:26.616994 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-cadaf518-824f-4437-a217-1e9458b4b584 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 08:23:26.616994 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:26.616994 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:26.616994 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:26.616994 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:26.616994 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:26.616994 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:26.616994 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:26.616994 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:26.616994 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:26.616994 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:26.616994 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:26.616994 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 08:23:26.616994 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 08:23:26.616994 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 08:23:26.616994 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 08:23:26.617916 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 08:23:26.617916 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 08:23:26.617916 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 08:23:26.617916 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 08:23:26.617916 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:26.617916 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:26.617916 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:26.617916 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:26.617916 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:26.617916 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:26.617916 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:26.617916 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:26.617916 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:26.617916 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:26.617916 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:26.617916 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:26.618721 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:26.618721 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:26.618721 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:26.618721 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:26.618721 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 08:23:26.618721 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 08:23:26.618721 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 08:23:26.618721 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 08:23:26.618721 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 08:23:26.618721 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 08:23:26.618721 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 08:23:26.618721 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:23:26.618721 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 211/441] 10.222.0.57 () {56 vars in 1114 bytes} [Wed Aug 10 08:23:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:26.619585 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59237473-b42a-48fc-9d10-b91eeef68e26 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:26.619876 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59237473-b42a-48fc-9d10-b91eeef68e26 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:26.620127 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59237473-b42a-48fc-9d10-b91eeef68e26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:26.637908 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-59237473-b42a-48fc-9d10-b91eeef68e26 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:23:26.639103 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-59237473-b42a-48fc-9d10-b91eeef68e26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:23:26.661700 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:26.704007 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f618b2e8-b74e-4118-9b48-2199bb480fdc None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:26.704982 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 212/442] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:23:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 08:23:26.705807 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59237473-b42a-48fc-9d10-b91eeef68e26 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:26.739741 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7c2a88-3853-4762-832d-21209ef3d6fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:26.740397 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7c2a88-3853-4762-832d-21209ef3d6fa None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:26.740678 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7c2a88-3853-4762-832d-21209ef3d6fa None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:26.740975 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7c2a88-3853-4762-832d-21209ef3d6fa None admin] PATH_INFO: `/v3/groups/db43b5c3821f4fda98bfe4343e15a822` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:26.742101 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7c2a88-3853-4762-832d-21209ef3d6fa None admin] RBAC: Authorizing `identity:get_group(group_id=db43b5c3821f4fda98bfe4343e15a822)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:26.743531 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7c2a88-3853-4762-832d-21209ef3d6fa None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:26.745119 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 231/443] 10.222.0.57 () {56 vars in 1180 bytes} [Wed Aug 10 08:23:26 2022] GET /identity/v3/groups/db43b5c3821f4fda98bfe4343e15a822 => generated 226 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:23:26.746201 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7c2a88-3853-4762-832d-21209ef3d6fa None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:26.750327 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:26.813451 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59237473-b42a-48fc-9d10-b91eeef68e26 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:26.846221 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7c2a88-3853-4762-832d-21209ef3d6fa None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:26.846958 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8885b4d-30bb-4f23-aa10-deb25fe5fa85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:26.847563 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8885b4d-30bb-4f23-aa10-deb25fe5fa85 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:26.847843 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8885b4d-30bb-4f23-aa10-deb25fe5fa85 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:26.848118 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8885b4d-30bb-4f23-aa10-deb25fe5fa85 None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:26.848995 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8885b4d-30bb-4f23-aa10-deb25fe5fa85 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:26.849163 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8885b4d-30bb-4f23-aa10-deb25fe5fa85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:26.850550 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8885b4d-30bb-4f23-aa10-deb25fe5fa85 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:26.858613 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 213/444] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Aug 10 08:23:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:26.859402 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8885b4d-30bb-4f23-aa10-deb25fe5fa85 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:26.883786 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:26.885124 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8391e4cd-fe7a-4ac6-86a6-a2e94550cfdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:26.885773 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8391e4cd-fe7a-4ac6-86a6-a2e94550cfdf None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:26.886050 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8391e4cd-fe7a-4ac6-86a6-a2e94550cfdf None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:26.886317 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8391e4cd-fe7a-4ac6-86a6-a2e94550cfdf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:26.896472 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8391e4cd-fe7a-4ac6-86a6-a2e94550cfdf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 08:23:26.901623 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-8391e4cd-fe7a-4ac6-86a6-a2e94550cfdf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 08:23:26.901623 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:26.901623 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:26.901623 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:26.901623 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:26.901623 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:26.901623 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:26.901623 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:26.901623 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:26.901623 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:26.901623 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:26.901623 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:26.901623 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 08:23:26.901623 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 08:23:26.901623 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 08:23:26.901623 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 08:23:26.902701 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 08:23:26.902701 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 08:23:26.902701 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 08:23:26.902701 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 08:23:26.902701 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:26.902701 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:26.902701 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:26.902701 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:26.902701 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:26.902701 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:26.902701 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:26.902701 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:26.902701 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:26.902701 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:26.902701 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:26.902701 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:26.903504 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:26.903504 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:26.903504 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:26.903504 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:26.903504 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 08:23:26.903504 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 08:23:26.903504 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 08:23:26.903504 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 08:23:26.903504 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 08:23:26.903504 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 08:23:26.903504 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 08:23:26.903504 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:23:26.903504 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 232/445] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Aug 10 08:23:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:26.904274 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8391e4cd-fe7a-4ac6-86a6-a2e94550cfdf None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:26.962549 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8885b4d-30bb-4f23-aa10-deb25fe5fa85 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:26.965077 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:26.996543 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee565116-5f5a-40ec-b1e3-5ce7e2c81b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:26.997193 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee565116-5f5a-40ec-b1e3-5ce7e2c81b89 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:26.997476 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee565116-5f5a-40ec-b1e3-5ce7e2c81b89 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:26.997751 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee565116-5f5a-40ec-b1e3-5ce7e2c81b89 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:26.998475 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee565116-5f5a-40ec-b1e3-5ce7e2c81b89 None admin] RBAC: Adding query filter params (group.id=db43b5c3821f4fda98bfe4343e15a822, role.id=44b64db484f24a46bf0b7ced46b519bf, scope.project.id=201dfacf93654f63a4732891e33d5039) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:26.998786 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee565116-5f5a-40ec-b1e3-5ce7e2c81b89 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:27.000220 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee565116-5f5a-40ec-b1e3-5ce7e2c81b89 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:27.002595 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8391e4cd-fe7a-4ac6-86a6-a2e94550cfdf None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:27.006602 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 214/446] 10.222.0.57 () {56 vars in 1376 bytes} [Wed Aug 10 08:23:26 2022] GET /identity/v3/role_assignments?group.id=db43b5c3821f4fda98bfe4343e15a822&scope.project.id=201dfacf93654f63a4732891e33d5039&role.id=44b64db484f24a46bf0b7ced46b519bf => generated 611 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:27.007388 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee565116-5f5a-40ec-b1e3-5ce7e2c81b89 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:27.039923 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67deaa7e-407a-40d2-9eb1-17616da8ec26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:27.040549 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67deaa7e-407a-40d2-9eb1-17616da8ec26 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:27.040830 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67deaa7e-407a-40d2-9eb1-17616da8ec26 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:27.041152 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67deaa7e-407a-40d2-9eb1-17616da8ec26 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:27.117378 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67deaa7e-407a-40d2-9eb1-17616da8ec26 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:27.118794 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67deaa7e-407a-40d2-9eb1-17616da8ec26 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:27.122891 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:27.141478 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee565116-5f5a-40ec-b1e3-5ce7e2c81b89 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:27.182000 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c7b63d-5a55-4811-b7d9-f1c39a93ba82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:27.182613 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c7b63d-5a55-4811-b7d9-f1c39a93ba82 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:27.182889 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c7b63d-5a55-4811-b7d9-f1c39a93ba82 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:27.183156 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c7b63d-5a55-4811-b7d9-f1c39a93ba82 None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:27.183810 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c7b63d-5a55-4811-b7d9-f1c39a93ba82 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:27.184122 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c7b63d-5a55-4811-b7d9-f1c39a93ba82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:27.185546 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c7b63d-5a55-4811-b7d9-f1c39a93ba82 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:27.195187 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 215/447] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Aug 10 08:23:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:27.195786 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:27.201014 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c7b63d-5a55-4811-b7d9-f1c39a93ba82 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:27.236506 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 233/448] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Aug 10 08:23:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 08:23:27.243513 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67deaa7e-407a-40d2-9eb1-17616da8ec26 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:27.332255 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c7b63d-5a55-4811-b7d9-f1c39a93ba82 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:27.371929 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67deaa7e-407a-40d2-9eb1-17616da8ec26 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:27.372396 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb49a14-9e43-41b0-ae23-53eb423fe3b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:27.373067 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb49a14-9e43-41b0-ae23-53eb423fe3b6 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:27.373351 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb49a14-9e43-41b0-ae23-53eb423fe3b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:27.373620 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb49a14-9e43-41b0-ae23-53eb423fe3b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:27.389727 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:27.408589 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02deeecc-f6a4-4ce2-989b-70533779d527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:27.409255 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02deeecc-f6a4-4ce2-989b-70533779d527 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:27.409532 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02deeecc-f6a4-4ce2-989b-70533779d527 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:27.409801 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02deeecc-f6a4-4ce2-989b-70533779d527 None admin] PATH_INFO: `/v3/projects/201dfacf93654f63a4732891e33d5039` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:27.411562 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02deeecc-f6a4-4ce2-989b-70533779d527 None admin] RBAC: Authorizing `identity:get_project(project_id=201dfacf93654f63a4732891e33d5039)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:27.413145 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02deeecc-f6a4-4ce2-989b-70533779d527 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:27.414934 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 234/449] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Aug 10 08:23:27 2022] GET /identity/v3/projects/201dfacf93654f63a4732891e33d5039 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:23:27.420408 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02deeecc-f6a4-4ce2-989b-70533779d527 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:27.444961 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb49a14-9e43-41b0-ae23-53eb423fe3b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:27.446424 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb49a14-9e43-41b0-ae23-53eb423fe3b6 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:27.452010 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:27.518757 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02deeecc-f6a4-4ce2-989b-70533779d527 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:27.544337 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 216/450] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Aug 10 08:23:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 10 08:23:27.549843 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb49a14-9e43-41b0-ae23-53eb423fe3b6 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:27.553422 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529282bb-18ad-4337-8680-c49d207f82ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:27.554045 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529282bb-18ad-4337-8680-c49d207f82ad None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:27.554328 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529282bb-18ad-4337-8680-c49d207f82ad None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:27.554598 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529282bb-18ad-4337-8680-c49d207f82ad None admin] PATH_INFO: `/v3/groups/db43b5c3821f4fda98bfe4343e15a822` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:27.555711 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529282bb-18ad-4337-8680-c49d207f82ad None admin] RBAC: Authorizing `identity:get_group(group_id=db43b5c3821f4fda98bfe4343e15a822)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:27.557264 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529282bb-18ad-4337-8680-c49d207f82ad None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:27.558852 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 235/451] 10.222.0.57 () {56 vars in 1180 bytes} [Wed Aug 10 08:23:27 2022] GET /identity/v3/groups/db43b5c3821f4fda98bfe4343e15a822 => generated 226 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:23:27.564547 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529282bb-18ad-4337-8680-c49d207f82ad None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:27.594948 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:27.646578 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb49a14-9e43-41b0-ae23-53eb423fe3b6 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:27.664155 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:27.666948 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529282bb-18ad-4337-8680-c49d207f82ad None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:27.680411 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200964a6-8581-4391-b61f-f572d487f854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:27.681082 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200964a6-8581-4391-b61f-f572d487f854 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:27.681367 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200964a6-8581-4391-b61f-f572d487f854 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:27.681639 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200964a6-8581-4391-b61f-f572d487f854 None admin] PATH_INFO: `/v3/projects/fe95c65d461a414fa25f77dc13eab47b` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:27.683286 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200964a6-8581-4391-b61f-f572d487f854 None admin] RBAC: Authorizing `identity:get_project(project_id=fe95c65d461a414fa25f77dc13eab47b)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:27.684353 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200964a6-8581-4391-b61f-f572d487f854 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:27.686247 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 217/452] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Aug 10 08:23:27 2022] GET /identity/v3/projects/fe95c65d461a414fa25f77dc13eab47b => generated 344 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:23:27.691771 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200964a6-8581-4391-b61f-f572d487f854 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:27.699826 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3545cf77-91e4-43a1-8351-4fdc3dd902e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:27.700416 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3545cf77-91e4-43a1-8351-4fdc3dd902e7 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:27.700687 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3545cf77-91e4-43a1-8351-4fdc3dd902e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:27.700981 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3545cf77-91e4-43a1-8351-4fdc3dd902e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:27.701699 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3545cf77-91e4-43a1-8351-4fdc3dd902e7 None admin] RBAC: Adding query filter params (group.id=db43b5c3821f4fda98bfe4343e15a822, role.id=ccfc8d3fdf2d458299df29001278930f, scope.project.id=201dfacf93654f63a4732891e33d5039) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:27.701999 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3545cf77-91e4-43a1-8351-4fdc3dd902e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:27.703504 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3545cf77-91e4-43a1-8351-4fdc3dd902e7 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:27.709632 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 236/453] 10.222.0.57 () {56 vars in 1376 bytes} [Wed Aug 10 08:23:27 2022] GET /identity/v3/role_assignments?group.id=db43b5c3821f4fda98bfe4343e15a822&scope.project.id=201dfacf93654f63a4732891e33d5039&role.id=ccfc8d3fdf2d458299df29001278930f => generated 611 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:27.805889 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200964a6-8581-4391-b61f-f572d487f854 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:27.821126 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:27.842322 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ad9012-2301-4f8c-867f-0cb3be13d751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:27.842960 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ad9012-2301-4f8c-867f-0cb3be13d751 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:27.843234 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ad9012-2301-4f8c-867f-0cb3be13d751 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:27.843499 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ad9012-2301-4f8c-867f-0cb3be13d751 None admin] PATH_INFO: `/v3/groups/a38ec50e49fb4028871d1d6f28421e35` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:27.845028 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ad9012-2301-4f8c-867f-0cb3be13d751 None admin] RBAC: Authorizing `identity:get_group(group_id=a38ec50e49fb4028871d1d6f28421e35)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:27.846464 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ad9012-2301-4f8c-867f-0cb3be13d751 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:27.848048 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 218/454] 10.222.0.57 () {56 vars in 1180 bytes} [Wed Aug 10 08:23:27 2022] GET /identity/v3/groups/a38ec50e49fb4028871d1d6f28421e35 => generated 229 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:23:27.854746 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3545cf77-91e4-43a1-8351-4fdc3dd902e7 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:27.956331 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3545cf77-91e4-43a1-8351-4fdc3dd902e7 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:27.993968 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddb304c-e6d1-46cf-a397-f8628d92aef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:27.994564 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddb304c-e6d1-46cf-a397-f8628d92aef8 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:27.994842 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddb304c-e6d1-46cf-a397-f8628d92aef8 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:27.995127 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddb304c-e6d1-46cf-a397-f8628d92aef8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:27.996172 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddb304c-e6d1-46cf-a397-f8628d92aef8 None admin] RBAC: Adding query filter params (group.id=a38ec50e49fb4028871d1d6f28421e35, role.id=ec3751692d9c4ecf81b4a71c936c46be, scope.project.id=fe95c65d461a414fa25f77dc13eab47b) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:27.996496 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddb304c-e6d1-46cf-a397-f8628d92aef8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:27.998025 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddb304c-e6d1-46cf-a397-f8628d92aef8 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:28.004013 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 237/455] 10.222.0.57 () {56 vars in 1376 bytes} [Wed Aug 10 08:23:27 2022] GET /identity/v3/role_assignments?group.id=a38ec50e49fb4028871d1d6f28421e35&scope.project.id=fe95c65d461a414fa25f77dc13eab47b&role.id=ec3751692d9c4ecf81b4a71c936c46be => generated 611 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:23:30.679540 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf8e7ee-a873-4ada-8fff-55f9fef8535d None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:30.681614 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf8e7ee-a873-4ada-8fff-55f9fef8535d None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:30.681614 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf8e7ee-a873-4ada-8fff-55f9fef8535d None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:30.681801 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 219/456] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:23:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:23:30.694581 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0735dc53-bd41-4f19-beb7-3308c620384f None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:30.695056 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0735dc53-bd41-4f19-beb7-3308c620384f None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:30.695131 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0735dc53-bd41-4f19-beb7-3308c620384f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:30.717895 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-0735dc53-bd41-4f19-beb7-3308c620384f None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:23:30.719112 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-0735dc53-bd41-4f19-beb7-3308c620384f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:23:30.726291 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b461dc76-d799-4001-b088-bab982d4a635 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:30.726581 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b461dc76-d799-4001-b088-bab982d4a635 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:30.726860 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b461dc76-d799-4001-b088-bab982d4a635 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:30.728452 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 220/457] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:23:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:23:30.733075 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c67653c-f3b6-4310-88db-d3e1a640d01a None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:30.733713 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c67653c-f3b6-4310-88db-d3e1a640d01a None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:30.734199 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c67653c-f3b6-4310-88db-d3e1a640d01a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:30.764035 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-3c67653c-f3b6-4310-88db-d3e1a640d01a None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:23:30.765668 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-3c67653c-f3b6-4310-88db-d3e1a640d01a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:23:30.775593 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:30.838083 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 238/458] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:23:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 08:23:30.839422 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e242d45b-db9b-4557-9f3c-c79c32d6fc58 None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:30.839707 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e242d45b-db9b-4557-9f3c-c79c32d6fc58 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:30.839979 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e242d45b-db9b-4557-9f3c-c79c32d6fc58 None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:30.841702 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 239/459] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:23:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:23:30.849894 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32102edb-8f58-41a2-8459-4c637b64ff82 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:30.850174 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32102edb-8f58-41a2-8459-4c637b64ff82 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:30.850445 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32102edb-8f58-41a2-8459-4c637b64ff82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:30.868246 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 221/460] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:23:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 08:23:30.868496 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-32102edb-8f58-41a2-8459-4c637b64ff82 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:23:30.870613 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-32102edb-8f58-41a2-8459-4c637b64ff82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:23:30.872659 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c67653c-f3b6-4310-88db-d3e1a640d01a None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:30.966381 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 240/461] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:23:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 08:23:30.973727 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32102edb-8f58-41a2-8459-4c637b64ff82 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:30.997927 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c67653c-f3b6-4310-88db-d3e1a640d01a None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:31.015264 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:31.042829 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bb83d3-beb7-4d15-9239-1e49f183544f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:31.043813 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bb83d3-beb7-4d15-9239-1e49f183544f None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:31.044295 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bb83d3-beb7-4d15-9239-1e49f183544f None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:31.045254 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bb83d3-beb7-4d15-9239-1e49f183544f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:31.060263 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bb83d3-beb7-4d15-9239-1e49f183544f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:31.063438 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bb83d3-beb7-4d15-9239-1e49f183544f None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:31.082994 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-a4bb83d3-beb7-4d15-9239-1e49f183544f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:23:31.082994 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:31.082994 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:31.082994 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:31.082994 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:31.082994 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:31.082994 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:31.082994 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:31.082994 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:31.082994 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:31.082994 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:31.082994 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:31.082994 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 08:23:31.082994 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 08:23:31.082994 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 08:23:31.082994 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 08:23:31.084006 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:31.084006 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:31.084006 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:31.084006 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:31.084006 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:31.084006 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:31.084006 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:31.084006 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:31.084006 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:31.084006 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:31.084006 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:31.084006 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:31.084006 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:31.084006 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:31.084006 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:31.084006 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:31.084006 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 08:23:31.085424 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 08:23:31.085424 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:23:31.085424 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:23:31.086036 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:31.093346 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 222/462] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:23:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 08:23:31.093346 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bb83d3-beb7-4d15-9239-1e49f183544f None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:31.104179 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32102edb-8f58-41a2-8459-4c637b64ff82 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:31.140308 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b749676b-e1cd-4982-b775-e1ea205868f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:31.141235 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b749676b-e1cd-4982-b775-e1ea205868f9 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:31.141515 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b749676b-e1cd-4982-b775-e1ea205868f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:31.141788 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b749676b-e1cd-4982-b775-e1ea205868f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:31.152502 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b749676b-e1cd-4982-b775-e1ea205868f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:31.153975 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b749676b-e1cd-4982-b775-e1ea205868f9 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:31.168485 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-b749676b-e1cd-4982-b775-e1ea205868f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:23:31.168485 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:31.168485 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:31.168485 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:31.168485 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:31.168485 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:31.168485 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:31.168485 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:31.168485 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:31.168485 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:31.168485 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:31.168485 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:31.168485 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 08:23:31.168485 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 08:23:31.168485 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 08:23:31.168485 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 08:23:31.169532 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:31.169532 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:31.169532 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:31.169532 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:31.169532 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:31.169532 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:31.169532 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:31.169532 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:31.169532 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:31.169532 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:31.169532 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:31.169532 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:31.169532 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:31.169532 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:31.169532 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:31.169532 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:31.169532 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 08:23:31.170395 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 08:23:31.170395 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:23:31.170395 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:23:31.170395 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 241/463] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:23:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:31.170395 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b749676b-e1cd-4982-b775-e1ea205868f9 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:31.209975 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bb83d3-beb7-4d15-9239-1e49f183544f None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:31.245235 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6ba19a-5428-4f68-aa7d-3d223be8d57e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:31.246049 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6ba19a-5428-4f68-aa7d-3d223be8d57e None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:31.246330 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6ba19a-5428-4f68-aa7d-3d223be8d57e None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:31.246603 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6ba19a-5428-4f68-aa7d-3d223be8d57e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:31.257969 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6ba19a-5428-4f68-aa7d-3d223be8d57e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:31.259443 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6ba19a-5428-4f68-aa7d-3d223be8d57e None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:31.274224 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-0e6ba19a-5428-4f68-aa7d-3d223be8d57e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:23:31.274224 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:31.274224 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:31.274224 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:31.274224 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:31.274224 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:31.274224 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:31.274224 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:31.274224 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:31.274224 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:31.274224 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:31.274224 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:31.274224 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 08:23:31.274224 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 08:23:31.274224 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 08:23:31.274224 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 08:23:31.275375 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:31.275375 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:31.275375 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:31.275375 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:31.275375 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:31.275375 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:31.275375 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:31.275375 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:31.275375 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:31.275375 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:31.275375 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:31.275375 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:31.275375 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:31.275375 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:31.275375 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:31.275375 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:31.275375 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 08:23:31.276471 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 08:23:31.276471 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:23:31.276471 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:23:31.276471 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 223/464] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:23:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:31.276769 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc109052-6263-463f-99b3-7c606ca572a8 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:31.276857 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc109052-6263-463f-99b3-7c606ca572a8 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:31.277265 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc109052-6263-463f-99b3-7c606ca572a8 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:31.277956 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b749676b-e1cd-4982-b775-e1ea205868f9 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:31.282512 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 224/465] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:23:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:23:31.282512 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc109052-6263-463f-99b3-7c606ca572a8 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:31.302696 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:31.329421 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9de89a0-c2bc-4b59-adf4-a6f8be4ab4a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:31.330312 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9de89a0-c2bc-4b59-adf4-a6f8be4ab4a5 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:31.330596 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9de89a0-c2bc-4b59-adf4-a6f8be4ab4a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:31.330870 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9de89a0-c2bc-4b59-adf4-a6f8be4ab4a5 None admin] PATH_INFO: `/v3/domains` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:31.331690 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9de89a0-c2bc-4b59-adf4-a6f8be4ab4a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:31.332376 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9de89a0-c2bc-4b59-adf4-a6f8be4ab4a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:31.334202 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9de89a0-c2bc-4b59-adf4-a6f8be4ab4a5 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:31.353966 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:31.362919 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 242/466] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Aug 10 08:23:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:31.362919 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9de89a0-c2bc-4b59-adf4-a6f8be4ab4a5 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:31.390642 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc109052-6263-463f-99b3-7c606ca572a8 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:31.431972 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d24dba-1407-487c-9072-e6ae95cea08b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:31.432608 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d24dba-1407-487c-9072-e6ae95cea08b None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:31.432861 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d24dba-1407-487c-9072-e6ae95cea08b None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:31.433164 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d24dba-1407-487c-9072-e6ae95cea08b None admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:31.433865 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d24dba-1407-487c-9072-e6ae95cea08b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:31.434110 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d24dba-1407-487c-9072-e6ae95cea08b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:31.435481 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d24dba-1407-487c-9072-e6ae95cea08b None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:31.453800 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 225/467] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Aug 10 08:23:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:31.453800 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f343685-2306-4008-9c7e-d2cc4b858e66 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:31.453800 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f343685-2306-4008-9c7e-d2cc4b858e66 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:31.453800 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f343685-2306-4008-9c7e-d2cc4b858e66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:31.470613 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-8f343685-2306-4008-9c7e-d2cc4b858e66 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:23:31.471809 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-8f343685-2306-4008-9c7e-d2cc4b858e66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:23:31.482408 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9de89a0-c2bc-4b59-adf4-a6f8be4ab4a5 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:31.522340 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d06d5c-c01f-44d8-be7e-2d24f8a80c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:31.522340 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d06d5c-c01f-44d8-be7e-2d24f8a80c64 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:31.522340 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d06d5c-c01f-44d8-be7e-2d24f8a80c64 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:31.522340 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d06d5c-c01f-44d8-be7e-2d24f8a80c64 None admin] PATH_INFO: `/v3/domains` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:31.522765 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d06d5c-c01f-44d8-be7e-2d24f8a80c64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:31.522765 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d06d5c-c01f-44d8-be7e-2d24f8a80c64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:31.522765 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d06d5c-c01f-44d8-be7e-2d24f8a80c64 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:31.538034 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 243/468] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Aug 10 08:23:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:31.538034 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d06d5c-c01f-44d8-be7e-2d24f8a80c64 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:31.549373 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 226/469] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:23:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 08:23:31.561884 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f343685-2306-4008-9c7e-d2cc4b858e66 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:31.572074 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:31.605062 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:31.649462 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d06d5c-c01f-44d8-be7e-2d24f8a80c64 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:31.670152 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f343685-2306-4008-9c7e-d2cc4b858e66 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:31.685879 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c3dbc0-a31d-4eaf-bb76-92e7d7a34f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:31.686627 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c3dbc0-a31d-4eaf-bb76-92e7d7a34f5f None admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:31.686908 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c3dbc0-a31d-4eaf-bb76-92e7d7a34f5f None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:31.687177 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c3dbc0-a31d-4eaf-bb76-92e7d7a34f5f None admin] PATH_INFO: `/v3/users` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:31.687814 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c3dbc0-a31d-4eaf-bb76-92e7d7a34f5f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:31.689277 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c3dbc0-a31d-4eaf-bb76-92e7d7a34f5f None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:31.709335 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6055f8bf-d4ee-495b-abf0-e720942ba699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:31.709923 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6055f8bf-d4ee-495b-abf0-e720942ba699 None admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:31.710208 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6055f8bf-d4ee-495b-abf0-e720942ba699 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:31.710535 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6055f8bf-d4ee-495b-abf0-e720942ba699 None admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:31.715507 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6055f8bf-d4ee-495b-abf0-e720942ba699 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:31.715507 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6055f8bf-d4ee-495b-abf0-e720942ba699 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:31.805190 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 244/470] 10.222.0.57 () {60 vars in 1132 bytes} [Wed Aug 10 08:23:31 2022] POST /identity/v3/users => generated 248 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:23:31.805190 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c3dbc0-a31d-4eaf-bb76-92e7d7a34f5f None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:31.832635 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 227/471] 10.222.0.57 () {60 vars in 1132 bytes} [Wed Aug 10 08:23:31 2022] POST /identity/v3/users => generated 248 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:23:31.834400 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6055f8bf-d4ee-495b-abf0-e720942ba699 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:31.936669 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:31.967319 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c3dbc0-a31d-4eaf-bb76-92e7d7a34f5f None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:31.989304 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:31.993765 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6055f8bf-d4ee-495b-abf0-e720942ba699 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:32.010490 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80cc321-9a40-4fa5-877a-c2af7d9ee7f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:32.011552 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80cc321-9a40-4fa5-877a-c2af7d9ee7f9 None admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:32.011845 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80cc321-9a40-4fa5-877a-c2af7d9ee7f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:32.012115 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80cc321-9a40-4fa5-877a-c2af7d9ee7f9 None admin] PATH_INFO: `/v3/users` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:32.012754 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80cc321-9a40-4fa5-877a-c2af7d9ee7f9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:32.014281 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80cc321-9a40-4fa5-877a-c2af7d9ee7f9 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:32.035988 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7d5e87-5061-46c4-97f5-937bf04d1da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:32.043045 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d5e87-5061-46c4-97f5-937bf04d1da2 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:32.043045 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d5e87-5061-46c4-97f5-937bf04d1da2 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:32.043045 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d5e87-5061-46c4-97f5-937bf04d1da2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:32.048243 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7d5e87-5061-46c4-97f5-937bf04d1da2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:32.049786 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7d5e87-5061-46c4-97f5-937bf04d1da2 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:32.065985 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-8c7d5e87-5061-46c4-97f5-937bf04d1da2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:23:32.065985 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:32.065985 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:32.065985 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:32.065985 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:32.065985 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:32.065985 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:32.065985 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:32.065985 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:32.065985 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:32.065985 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:32.065985 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:32.065985 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 08:23:32.065985 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 08:23:32.065985 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 08:23:32.065985 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 08:23:32.067793 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:32.067793 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:32.067793 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:32.067793 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:32.067793 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:32.067793 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:32.067793 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:32.067793 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:32.067793 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:32.067793 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:32.067793 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:32.067793 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:32.067793 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:32.067793 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:32.067793 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:32.067793 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:32.067793 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 08:23:32.068732 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 08:23:32.068732 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:23:32.068732 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:23:32.068732 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 228/472] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:23:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:32.068732 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514b15a3-3342-4bcd-b43c-6792db549bf0 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:32.068732 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514b15a3-3342-4bcd-b43c-6792db549bf0 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:32.069123 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514b15a3-3342-4bcd-b43c-6792db549bf0 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:32.070214 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 229/473] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:23:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:23:32.076287 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf831796-b5fc-402d-a5aa-7877d280b3f8 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:32.076712 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf831796-b5fc-402d-a5aa-7877d280b3f8 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:32.077021 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf831796-b5fc-402d-a5aa-7877d280b3f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:32.096484 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-bf831796-b5fc-402d-a5aa-7877d280b3f8 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:23:32.097703 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-bf831796-b5fc-402d-a5aa-7877d280b3f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:23:32.110037 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 245/474] 10.222.0.57 () {60 vars in 1132 bytes} [Wed Aug 10 08:23:31 2022] POST /identity/v3/users => generated 246 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:23:32.111586 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80cc321-9a40-4fa5-877a-c2af7d9ee7f9 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:32.193155 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 230/475] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:23:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 08:23:32.213380 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf831796-b5fc-402d-a5aa-7877d280b3f8 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:32.268959 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:32.281876 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80cc321-9a40-4fa5-877a-c2af7d9ee7f9 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:32.320587 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102d65f3-cd5e-445a-a14e-458e46497ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:32.321581 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102d65f3-cd5e-445a-a14e-458e46497ee9 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:32.322074 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102d65f3-cd5e-445a-a14e-458e46497ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:32.322414 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102d65f3-cd5e-445a-a14e-458e46497ee9 None admin] PATH_INFO: `/v3/domains` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:32.323138 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102d65f3-cd5e-445a-a14e-458e46497ee9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:32.323526 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102d65f3-cd5e-445a-a14e-458e46497ee9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:32.325871 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102d65f3-cd5e-445a-a14e-458e46497ee9 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:32.335825 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf831796-b5fc-402d-a5aa-7877d280b3f8 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:32.342217 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 246/476] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Aug 10 08:23:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:32.349443 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102d65f3-cd5e-445a-a14e-458e46497ee9 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:32.381371 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32416458-1cb6-4268-8b63-3b203784f415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:32.382963 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32416458-1cb6-4268-8b63-3b203784f415 None admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:32.383423 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32416458-1cb6-4268-8b63-3b203784f415 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:32.383927 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32416458-1cb6-4268-8b63-3b203784f415 None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:32.385681 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:32.386584 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32416458-1cb6-4268-8b63-3b203784f415 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:32.388220 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32416458-1cb6-4268-8b63-3b203784f415 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:32.420427 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.sql.core [None req-32416458-1cb6-4268-8b63-3b203784f415 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 10 08:23:32.420427 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 10 08:23:32.420427 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [parameters: {'id': 'ae0bb80a0ed34b3f9c6bca49ac98b043', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 10 08:23:32.420427 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80462) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 10 08:23:32.447267 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-32416458-1cb6-4268-8b63-3b203784f415 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 10 08:23:32.447267 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:32.447267 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 10 08:23:32.447267 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 10 08:23:32.447267 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 10 08:23:32.447267 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 10 08:23:32.447267 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 10 08:23:32.447267 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application result = self._query(query) Aug 10 08:23:32.447267 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 10 08:23:32.447267 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application conn.query(q) Aug 10 08:23:32.447267 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 10 08:23:32.447267 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 10 08:23:32.447267 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 10 08:23:32.447267 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application result.read() Aug 10 08:23:32.447267 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 10 08:23:32.448235 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 10 08:23:32.448235 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 10 08:23:32.448235 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 10 08:23:32.448235 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 10 08:23:32.448235 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 10 08:23:32.448235 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 10 08:23:32.448235 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 10 08:23:32.448235 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 10 08:23:32.448235 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:23:32.448235 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 10 08:23:32.448235 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:23:32.448235 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:32.448235 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 10 08:23:32.448235 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 10 08:23:32.448235 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 10 08:23:32.448235 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return ref.to_dict() Aug 10 08:23:32.448235 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 10 08:23:32.448235 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application next(self.gen) Aug 10 08:23:32.449305 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 10 08:23:32.449305 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application yield resource Aug 10 08:23:32.449305 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 10 08:23:32.449305 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application next(self.gen) Aug 10 08:23:32.449305 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 10 08:23:32.449305 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application self.session.rollback() Aug 10 08:23:32.449305 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 10 08:23:32.449305 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application self.force_reraise() Aug 10 08:23:32.449305 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 10 08:23:32.449305 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise self.value Aug 10 08:23:32.449305 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 10 08:23:32.449305 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 10 08:23:32.449305 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 10 08:23:32.449305 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application session.commit() Aug 10 08:23:32.449305 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 10 08:23:32.449305 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 10 08:23:32.449305 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 10 08:23:32.450173 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application self._prepare_impl() Aug 10 08:23:32.450173 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 10 08:23:32.450173 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application self.session.flush() Aug 10 08:23:32.450173 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 10 08:23:32.450173 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application self._flush(objects) Aug 10 08:23:32.450173 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 10 08:23:32.450173 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 10 08:23:32.450173 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 10 08:23:32.450173 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application compat.raise_( Aug 10 08:23:32.450173 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 10 08:23:32.450173 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception Aug 10 08:23:32.450173 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 10 08:23:32.450173 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application flush_context.execute() Aug 10 08:23:32.450173 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 10 08:23:32.450173 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rec.execute(self) Aug 10 08:23:32.450173 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 10 08:23:32.450173 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 10 08:23:32.451034 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 10 08:23:32.451034 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 10 08:23:32.451034 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 10 08:23:32.451034 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 10 08:23:32.451034 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 10 08:23:32.451034 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 10 08:23:32.451034 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 10 08:23:32.451034 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 10 08:23:32.451034 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 10 08:23:32.451034 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 10 08:23:32.451034 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 10 08:23:32.451034 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 10 08:23:32.451034 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 10 08:23:32.451034 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 10 08:23:32.451034 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 10 08:23:32.451034 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception Aug 10 08:23:32.453153 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 10 08:23:32.453153 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 10 08:23:32.453153 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 10 08:23:32.453153 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 10 08:23:32.453153 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 10 08:23:32.453153 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application result = self._query(query) Aug 10 08:23:32.453153 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 10 08:23:32.453153 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application conn.query(q) Aug 10 08:23:32.453153 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 10 08:23:32.453153 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 10 08:23:32.453153 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 10 08:23:32.453153 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application result.read() Aug 10 08:23:32.453153 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 10 08:23:32.453153 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 10 08:23:32.453153 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 10 08:23:32.453153 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 10 08:23:32.459508 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 10 08:23:32.459508 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 10 08:23:32.459508 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 10 08:23:32.459508 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 10 08:23:32.459508 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 10 08:23:32.459508 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 10 08:23:32.459508 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [parameters: {'id': 'ae0bb80a0ed34b3f9c6bca49ac98b043', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 10 08:23:32.459508 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 10 08:23:32.459508 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:23:32.459508 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 08:23:32.459508 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:23:32.459508 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:32.459508 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:32.459508 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:32.459508 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:32.459508 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:32.461475 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:32.461475 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:32.461475 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:32.461475 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:32.461475 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:32.461475 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:32.461475 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 10 08:23:32.461475 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 10 08:23:32.461475 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:32.461475 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:32.461475 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 10 08:23:32.461475 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 10 08:23:32.461475 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 10 08:23:32.461475 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 10 08:23:32.461475 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 10 08:23:32.461475 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:23:32.463459 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 231/477] 10.222.0.57 () {60 vars in 1131 bytes} [Wed Aug 10 08:23:32 2022] POST /identity/v3/roles => generated 144 bytes in 238 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 10 08:23:32.467231 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32416458-1cb6-4268-8b63-3b203784f415 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:32.487116 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102d65f3-cd5e-445a-a14e-458e46497ee9 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:32.540889 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5614fde-8337-4b81-a1f1-77a95a15d081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:32.542061 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5614fde-8337-4b81-a1f1-77a95a15d081 None admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:32.542824 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5614fde-8337-4b81-a1f1-77a95a15d081 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:32.543380 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5614fde-8337-4b81-a1f1-77a95a15d081 None admin] PATH_INFO: `/v3/users` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:32.544225 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5614fde-8337-4b81-a1f1-77a95a15d081 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:32.546403 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5614fde-8337-4b81-a1f1-77a95a15d081 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:32.604967 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32416458-1cb6-4268-8b63-3b203784f415 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:32.615687 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:32.631496 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 247/478] 10.222.0.57 () {60 vars in 1132 bytes} [Wed Aug 10 08:23:32 2022] POST /identity/v3/users => generated 249 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:23:32.654447 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c150472-82af-4f92-a3d3-24414049704d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:32.655484 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c150472-82af-4f92-a3d3-24414049704d None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:32.655976 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c150472-82af-4f92-a3d3-24414049704d None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:32.656420 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c150472-82af-4f92-a3d3-24414049704d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:32.669352 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c150472-82af-4f92-a3d3-24414049704d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 08:23:32.674942 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-3c150472-82af-4f92-a3d3-24414049704d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 08:23:32.674942 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:32.674942 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:32.674942 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:32.674942 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:32.674942 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:32.674942 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:32.674942 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:32.674942 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:32.674942 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:32.674942 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:32.674942 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:32.674942 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 08:23:32.674942 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 08:23:32.674942 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 08:23:32.674942 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 08:23:32.675955 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 08:23:32.675955 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 08:23:32.675955 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 08:23:32.675955 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 08:23:32.675955 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:32.675955 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:32.675955 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:32.675955 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:32.675955 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:32.675955 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:32.675955 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:32.675955 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:32.675955 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:32.675955 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:32.675955 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:32.675955 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:32.677069 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:32.677069 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:32.677069 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:32.677069 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:32.677069 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 08:23:32.677069 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 08:23:32.677069 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 08:23:32.677069 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 08:23:32.677069 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 08:23:32.677069 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 08:23:32.677069 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 08:23:32.677069 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:23:32.679593 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 232/479] 10.222.0.57 () {56 vars in 1114 bytes} [Wed Aug 10 08:23:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 08:23:32.685928 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5614fde-8337-4b81-a1f1-77a95a15d081 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:32.803886 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:32.819637 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5614fde-8337-4b81-a1f1-77a95a15d081 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:32.861896 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df358dc9-2071-4478-8bfa-2d1a6ddd6d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:32.862835 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df358dc9-2071-4478-8bfa-2d1a6ddd6d3e None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:32.863251 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df358dc9-2071-4478-8bfa-2d1a6ddd6d3e None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:32.863693 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df358dc9-2071-4478-8bfa-2d1a6ddd6d3e None admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:32.864528 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df358dc9-2071-4478-8bfa-2d1a6ddd6d3e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:32.865029 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df358dc9-2071-4478-8bfa-2d1a6ddd6d3e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:32.866613 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df358dc9-2071-4478-8bfa-2d1a6ddd6d3e None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:32.881351 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 248/480] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Aug 10 08:23:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:34.029294 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d99aadc-23b5-400a-ac54-ba7b31d7d010 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:34.029842 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d99aadc-23b5-400a-ac54-ba7b31d7d010 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:34.030113 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d99aadc-23b5-400a-ac54-ba7b31d7d010 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:34.031825 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 233/481] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:23:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:23:34.037394 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59224202-4873-42f3-a096-f346ecdcb042 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:34.037825 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59224202-4873-42f3-a096-f346ecdcb042 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:34.038010 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59224202-4873-42f3-a096-f346ecdcb042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:34.062683 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-59224202-4873-42f3-a096-f346ecdcb042 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:23:34.063881 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-59224202-4873-42f3-a096-f346ecdcb042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:23:34.142321 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 249/482] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:23:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 08:23:34.176608 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d99aadc-23b5-400a-ac54-ba7b31d7d010 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:34.291140 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d99aadc-23b5-400a-ac54-ba7b31d7d010 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:34.331387 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dcbd29-cf27-434f-977b-4fdc9df1c592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:34.332015 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dcbd29-cf27-434f-977b-4fdc9df1c592 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:34.332293 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dcbd29-cf27-434f-977b-4fdc9df1c592 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:34.332652 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dcbd29-cf27-434f-977b-4fdc9df1c592 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:34.349815 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dcbd29-cf27-434f-977b-4fdc9df1c592 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 08:23:34.354578 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-82dcbd29-cf27-434f-977b-4fdc9df1c592 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 08:23:34.354578 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:34.354578 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:34.354578 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:34.354578 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:34.354578 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:34.354578 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:34.354578 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:34.354578 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:34.354578 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:34.354578 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:34.354578 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:34.354578 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 08:23:34.354578 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 08:23:34.354578 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 08:23:34.354578 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 08:23:34.355443 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 08:23:34.355443 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 08:23:34.355443 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 08:23:34.355443 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 08:23:34.355443 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:34.355443 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:34.355443 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:34.355443 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:34.355443 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:34.355443 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:34.355443 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:34.355443 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:34.355443 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:34.355443 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:34.355443 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:34.355443 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:34.356260 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:34.356260 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:34.356260 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:34.356260 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:34.356260 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 08:23:34.356260 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 08:23:34.356260 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 08:23:34.356260 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 08:23:34.356260 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 08:23:34.356260 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 08:23:34.356260 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 08:23:34.356260 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:23:34.356260 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 234/483] 10.222.0.57 () {56 vars in 1102 bytes} [Wed Aug 10 08:23:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:34.361727 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59224202-4873-42f3-a096-f346ecdcb042 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:34.469448 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37a7087-bd0e-403a-820b-d1dbf5fecbf3 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:34.469788 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37a7087-bd0e-403a-820b-d1dbf5fecbf3 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:34.469990 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37a7087-bd0e-403a-820b-d1dbf5fecbf3 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:34.471576 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 235/484] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:23:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:23:34.477213 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292faa88-bb83-46f9-8792-c85e90479022 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:34.477494 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292faa88-bb83-46f9-8792-c85e90479022 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:34.477765 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292faa88-bb83-46f9-8792-c85e90479022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:34.488534 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:34.509173 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59224202-4873-42f3-a096-f346ecdcb042 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:34.513973 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-292faa88-bb83-46f9-8792-c85e90479022 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:23:34.515176 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-292faa88-bb83-46f9-8792-c85e90479022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:23:34.547022 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c8c918-41b8-409a-88ad-02f8f06e0dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:34.547556 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c8c918-41b8-409a-88ad-02f8f06e0dd4 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:34.547835 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c8c918-41b8-409a-88ad-02f8f06e0dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:34.548109 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c8c918-41b8-409a-88ad-02f8f06e0dd4 None admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:34.548782 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c8c918-41b8-409a-88ad-02f8f06e0dd4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:34.549116 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c8c918-41b8-409a-88ad-02f8f06e0dd4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:34.550522 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c8c918-41b8-409a-88ad-02f8f06e0dd4 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:34.559992 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 250/485] 10.222.0.57 () {56 vars in 1103 bytes} [Wed Aug 10 08:23:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:34.565349 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c8c918-41b8-409a-88ad-02f8f06e0dd4 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:34.576125 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:34.676210 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c8c918-41b8-409a-88ad-02f8f06e0dd4 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:34.706635 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 236/486] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:23:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 08:23:34.712975 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadbc108-78af-49a6-aeab-dbba3986262f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:34.713620 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadbc108-78af-49a6-aeab-dbba3986262f None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:34.713901 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadbc108-78af-49a6-aeab-dbba3986262f None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:34.714173 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadbc108-78af-49a6-aeab-dbba3986262f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:34.725436 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadbc108-78af-49a6-aeab-dbba3986262f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:34.726884 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadbc108-78af-49a6-aeab-dbba3986262f None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:34.728647 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:34.742346 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-cadbc108-78af-49a6-aeab-dbba3986262f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:23:34.742346 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:34.742346 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:34.742346 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:34.742346 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:34.742346 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:34.742346 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:34.742346 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:34.742346 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:34.742346 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:34.742346 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:34.742346 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:34.742346 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 08:23:34.742346 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 08:23:34.742346 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 08:23:34.742346 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 08:23:34.743249 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:34.743249 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:34.743249 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:34.743249 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:34.743249 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:34.743249 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:34.743249 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:34.743249 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:34.743249 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:34.743249 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:34.743249 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:34.743249 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:34.743249 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:34.743249 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:34.743249 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:34.743249 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:34.743249 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 08:23:34.744104 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 08:23:34.744104 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:23:34.744104 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:23:34.744104 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 251/487] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:23:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:34.749033 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadbc108-78af-49a6-aeab-dbba3986262f None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:34.749177 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292faa88-bb83-46f9-8792-c85e90479022 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:34.861102 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadbc108-78af-49a6-aeab-dbba3986262f None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:34.903259 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2936ea29-9584-47f2-a3b2-cd7b91ce8a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:34.903876 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2936ea29-9584-47f2-a3b2-cd7b91ce8a06 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:34.904153 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2936ea29-9584-47f2-a3b2-cd7b91ce8a06 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:34.904425 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2936ea29-9584-47f2-a3b2-cd7b91ce8a06 None admin] PATH_INFO: `/v3/domains` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:34.905097 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2936ea29-9584-47f2-a3b2-cd7b91ce8a06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:34.905399 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2936ea29-9584-47f2-a3b2-cd7b91ce8a06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:34.906954 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2936ea29-9584-47f2-a3b2-cd7b91ce8a06 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:34.932061 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 252/488] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Aug 10 08:23:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:23:34.934473 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e65c96a-b332-48bb-a1e2-abc63b677ba8 None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:34.935049 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e65c96a-b332-48bb-a1e2-abc63b677ba8 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:34.935493 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e65c96a-b332-48bb-a1e2-abc63b677ba8 None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:34.937583 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 253/489] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:23:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:23:34.938723 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e65c96a-b332-48bb-a1e2-abc63b677ba8 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:34.948390 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292faa88-bb83-46f9-8792-c85e90479022 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:34.992450 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d702e5-4ae6-4034-bbdf-9fb02d5a98d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:34.993658 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d702e5-4ae6-4034-bbdf-9fb02d5a98d7 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:34.994023 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d702e5-4ae6-4034-bbdf-9fb02d5a98d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:34.994409 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d702e5-4ae6-4034-bbdf-9fb02d5a98d7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:35.004003 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:35.007537 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d702e5-4ae6-4034-bbdf-9fb02d5a98d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 08:23:35.012997 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-25d702e5-4ae6-4034-bbdf-9fb02d5a98d7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 08:23:35.012997 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:35.012997 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:35.012997 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:35.012997 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:35.012997 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:35.012997 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:35.012997 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:35.012997 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:35.012997 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:35.012997 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:35.012997 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:35.012997 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 08:23:35.012997 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 08:23:35.012997 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 08:23:35.012997 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 08:23:35.021354 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 08:23:35.021354 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 08:23:35.021354 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 08:23:35.021354 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 08:23:35.021354 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:35.021354 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:35.021354 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:35.021354 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:35.021354 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:35.021354 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:35.021354 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:35.021354 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:35.021354 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:35.021354 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:35.021354 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:35.021354 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:35.022150 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:35.022150 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:35.022150 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:35.022150 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:35.022150 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 08:23:35.022150 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 08:23:35.022150 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 08:23:35.022150 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 08:23:35.022150 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 08:23:35.022150 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 08:23:35.022150 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 08:23:35.022150 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:23:35.022150 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 237/490] 10.222.0.57 () {56 vars in 1102 bytes} [Wed Aug 10 08:23:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:35.022150 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ac591e-4bf6-4bfb-a2a6-9c9198492c78 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:35.022878 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ac591e-4bf6-4bfb-a2a6-9c9198492c78 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:35.022878 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ac591e-4bf6-4bfb-a2a6-9c9198492c78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:35.042918 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-f8ac591e-4bf6-4bfb-a2a6-9c9198492c78 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:23:35.044867 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-f8ac591e-4bf6-4bfb-a2a6-9c9198492c78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:23:35.060644 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e65c96a-b332-48bb-a1e2-abc63b677ba8 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:35.064791 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:35.100416 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57e3c89-835b-4c03-bff8-830f97565821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:35.101172 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57e3c89-835b-4c03-bff8-830f97565821 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:35.101518 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57e3c89-835b-4c03-bff8-830f97565821 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:35.101850 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57e3c89-835b-4c03-bff8-830f97565821 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:35.110233 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57e3c89-835b-4c03-bff8-830f97565821 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:35.112011 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57e3c89-835b-4c03-bff8-830f97565821 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:35.127275 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-e57e3c89-835b-4c03-bff8-830f97565821 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 10 08:23:35.127275 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:35.127275 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:35.127275 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:35.127275 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:35.127275 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:35.127275 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:35.127275 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:35.127275 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:35.127275 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:35.127275 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:35.127275 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:35.127275 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 08:23:35.127275 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 08:23:35.127275 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 08:23:35.127275 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 08:23:35.128245 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:35.128245 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:35.128245 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 08:23:35.128245 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 08:23:35.128245 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 08:23:35.128245 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 08:23:35.128245 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:35.128245 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:35.128245 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:35.128245 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:35.128245 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:35.128245 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:35.128245 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:35.128245 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:35.128245 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:35.128245 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:35.128245 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:35.128245 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:35.129860 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:35.129860 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:35.129860 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 08:23:35.129860 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 08:23:35.129860 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 08:23:35.129860 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 08:23:35.129860 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 08:23:35.129860 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 08:23:35.129860 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 10 08:23:35.129860 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:23:35.129860 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 254/491] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Aug 10 08:23:34 2022] GET /identity/v3/users/glance => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:35.130516 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57e3c89-835b-4c03-bff8-830f97565821 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:35.140107 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 238/492] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:23:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 08:23:35.140107 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ac591e-4bf6-4bfb-a2a6-9c9198492c78 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:35.255013 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ac591e-4bf6-4bfb-a2a6-9c9198492c78 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:35.272066 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57e3c89-835b-4c03-bff8-830f97565821 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:35.288176 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:35.300026 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c49f67-852e-4c69-9301-e857cd364e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:35.309113 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c49f67-852e-4c69-9301-e857cd364e5b None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:35.309113 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c49f67-852e-4c69-9301-e857cd364e5b None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:35.309113 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c49f67-852e-4c69-9301-e857cd364e5b None admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:35.309113 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c49f67-852e-4c69-9301-e857cd364e5b None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:35.309113 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c49f67-852e-4c69-9301-e857cd364e5b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:35.309113 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c49f67-852e-4c69-9301-e857cd364e5b None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:35.309526 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6659125-e57a-437f-9375-5ca75372a2f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:35.310039 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6659125-e57a-437f-9375-5ca75372a2f0 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:35.310318 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6659125-e57a-437f-9375-5ca75372a2f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:35.310584 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6659125-e57a-437f-9375-5ca75372a2f0 None admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:35.311249 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6659125-e57a-437f-9375-5ca75372a2f0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:35.311552 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6659125-e57a-437f-9375-5ca75372a2f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:35.312943 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6659125-e57a-437f-9375-5ca75372a2f0 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:35.322626 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 255/493] 10.222.0.57 () {56 vars in 1103 bytes} [Wed Aug 10 08:23:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:35.323443 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6659125-e57a-437f-9375-5ca75372a2f0 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:35.354344 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 239/494] 10.222.0.57 () {56 vars in 1137 bytes} [Wed Aug 10 08:23:35 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:35.355823 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c49f67-852e-4c69-9301-e857cd364e5b None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:35.368226 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:35.452354 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6659125-e57a-437f-9375-5ca75372a2f0 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:35.491187 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a1f21d-be68-4887-ab8b-f0dcf6778f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:35.492259 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a1f21d-be68-4887-ab8b-f0dcf6778f8a None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:35.492668 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a1f21d-be68-4887-ab8b-f0dcf6778f8a None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:35.492953 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a1f21d-be68-4887-ab8b-f0dcf6778f8a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:35.494200 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c49f67-852e-4c69-9301-e857cd364e5b None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:35.506414 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a1f21d-be68-4887-ab8b-f0dcf6778f8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 08:23:35.512742 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-32a1f21d-be68-4887-ab8b-f0dcf6778f8a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 08:23:35.512742 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:35.512742 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:35.512742 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:35.512742 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:35.512742 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:35.512742 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:35.512742 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:35.512742 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:35.512742 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:35.512742 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:35.512742 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:35.512742 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 08:23:35.512742 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 08:23:35.512742 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 08:23:35.512742 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 08:23:35.513671 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 08:23:35.513671 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 08:23:35.513671 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 08:23:35.513671 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 08:23:35.513671 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:35.513671 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:35.513671 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:35.513671 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:35.513671 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:35.513671 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:35.513671 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:35.513671 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:35.513671 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:35.513671 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:35.513671 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:35.513671 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:35.519476 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:35.519476 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:35.519476 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:35.519476 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:35.519476 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 08:23:35.519476 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 08:23:35.519476 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 08:23:35.519476 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 08:23:35.519476 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 08:23:35.519476 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 08:23:35.519476 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 08:23:35.519476 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:23:35.519476 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 256/495] 10.222.0.57 () {56 vars in 1102 bytes} [Wed Aug 10 08:23:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:35.519476 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a1f21d-be68-4887-ab8b-f0dcf6778f8a None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:35.528180 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61a22ce-b71e-4efb-8a56-4f799ce67e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:35.528819 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61a22ce-b71e-4efb-8a56-4f799ce67e63 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:35.529130 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61a22ce-b71e-4efb-8a56-4f799ce67e63 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:35.529406 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61a22ce-b71e-4efb-8a56-4f799ce67e63 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:35.540577 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61a22ce-b71e-4efb-8a56-4f799ce67e63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:35.542113 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61a22ce-b71e-4efb-8a56-4f799ce67e63 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:35.556817 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-c61a22ce-b71e-4efb-8a56-4f799ce67e63 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:23:35.556817 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:35.556817 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:35.556817 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:35.556817 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:35.556817 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:35.556817 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:35.556817 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:35.556817 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:35.556817 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:35.556817 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:35.556817 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:35.556817 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 08:23:35.556817 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 08:23:35.556817 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 08:23:35.556817 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 08:23:35.557791 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:35.557791 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:35.557791 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:35.557791 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:35.557791 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:35.557791 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:35.557791 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:35.557791 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:35.557791 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:35.557791 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:35.557791 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:35.557791 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:35.557791 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:35.557791 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:35.557791 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:35.557791 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:35.557791 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 08:23:35.558663 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 08:23:35.558663 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:23:35.558663 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:23:35.558663 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 240/496] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:23:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:35.559159 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9a431b-51c4-4994-bbd0-6af8597d4120 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:35.559365 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9a431b-51c4-4994-bbd0-6af8597d4120 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:35.559637 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9a431b-51c4-4994-bbd0-6af8597d4120 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:35.564956 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 241/497] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:23:35 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:23:35.564956 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9a431b-51c4-4994-bbd0-6af8597d4120 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:35.568526 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:35.625890 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:35.637132 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a1f21d-be68-4887-ab8b-f0dcf6778f8a None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:35.668269 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9a431b-51c4-4994-bbd0-6af8597d4120 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:35.670754 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d56561-8be5-4a7f-93c5-dda38f85ad12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:35.671353 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d56561-8be5-4a7f-93c5-dda38f85ad12 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:35.671632 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d56561-8be5-4a7f-93c5-dda38f85ad12 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:35.671903 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d56561-8be5-4a7f-93c5-dda38f85ad12 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:35.701862 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6be2ae-92e3-4f26-8fa9-41ab5cee8982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:35.702481 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6be2ae-92e3-4f26-8fa9-41ab5cee8982 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:35.702726 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6be2ae-92e3-4f26-8fa9-41ab5cee8982 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:35.702991 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6be2ae-92e3-4f26-8fa9-41ab5cee8982 None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:35.703639 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6be2ae-92e3-4f26-8fa9-41ab5cee8982 None admin] RBAC: Adding query filter params (name=service) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:35.703934 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6be2ae-92e3-4f26-8fa9-41ab5cee8982 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:35.705336 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6be2ae-92e3-4f26-8fa9-41ab5cee8982 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:35.713349 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 242/498] 10.222.0.57 () {56 vars in 1103 bytes} [Wed Aug 10 08:23:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:35.714139 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6be2ae-92e3-4f26-8fa9-41ab5cee8982 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:35.739462 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d56561-8be5-4a7f-93c5-dda38f85ad12 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:35.741057 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d56561-8be5-4a7f-93c5-dda38f85ad12 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:35.775028 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:35.813890 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6be2ae-92e3-4f26-8fa9-41ab5cee8982 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:35.834128 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:35.840298 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 257/499] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Aug 10 08:23:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 08:23:35.841733 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daba0b27-2823-41db-9134-18a8c734dfe5 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:35.842011 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daba0b27-2823-41db-9134-18a8c734dfe5 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:35.842276 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daba0b27-2823-41db-9134-18a8c734dfe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:35.848571 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875f0643-2bd5-46c3-9dc9-44364b9a07f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:35.849202 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875f0643-2bd5-46c3-9dc9-44364b9a07f0 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:35.849483 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875f0643-2bd5-46c3-9dc9-44364b9a07f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:35.849752 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875f0643-2bd5-46c3-9dc9-44364b9a07f0 None admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:35.850392 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875f0643-2bd5-46c3-9dc9-44364b9a07f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:35.850695 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875f0643-2bd5-46c3-9dc9-44364b9a07f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:35.852043 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875f0643-2bd5-46c3-9dc9-44364b9a07f0 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:35.860530 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-daba0b27-2823-41db-9134-18a8c734dfe5 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:23:35.861736 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-daba0b27-2823-41db-9134-18a8c734dfe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:23:35.863594 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 243/500] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Aug 10 08:23:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:35.864835 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859adf49-4c65-4c59-a959-2eb35b06986e None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:35.865211 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859adf49-4c65-4c59-a959-2eb35b06986e None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:35.865489 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859adf49-4c65-4c59-a959-2eb35b06986e None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:35.866824 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 244/501] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:23:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:23:35.869708 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859adf49-4c65-4c59-a959-2eb35b06986e None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:35.930962 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 258/502] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:23:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 08:23:35.941291 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daba0b27-2823-41db-9134-18a8c734dfe5 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:35.981936 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859adf49-4c65-4c59-a959-2eb35b06986e None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:35.996138 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:36.015138 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f06b44-7eaf-4655-9031-d3c410c90952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:36.015760 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f06b44-7eaf-4655-9031-d3c410c90952 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:36.016041 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f06b44-7eaf-4655-9031-d3c410c90952 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:36.016315 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f06b44-7eaf-4655-9031-d3c410c90952 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:36.026837 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f06b44-7eaf-4655-9031-d3c410c90952 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:36.028287 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f06b44-7eaf-4655-9031-d3c410c90952 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:36.029995 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daba0b27-2823-41db-9134-18a8c734dfe5 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:36.043229 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-c3f06b44-7eaf-4655-9031-d3c410c90952 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:23:36.043229 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:36.043229 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:36.043229 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:36.043229 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:36.043229 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:36.043229 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:36.043229 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:36.043229 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:36.043229 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:36.043229 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:36.043229 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:36.043229 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 08:23:36.043229 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 08:23:36.043229 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 08:23:36.043229 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 08:23:36.044502 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:36.044502 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:36.044502 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:36.044502 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:36.044502 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:36.044502 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:36.044502 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:36.044502 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:36.044502 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:36.044502 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:36.044502 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:36.044502 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:36.044502 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:36.044502 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:36.044502 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:36.044502 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:36.044502 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 08:23:36.045851 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 08:23:36.045851 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:23:36.045851 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:23:36.045851 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 245/503] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:23:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:36.045851 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f06b44-7eaf-4655-9031-d3c410c90952 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:36.067816 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a097ec9-c05e-4fb4-918f-4cbb6431f16d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:36.068495 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a097ec9-c05e-4fb4-918f-4cbb6431f16d None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:36.068772 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a097ec9-c05e-4fb4-918f-4cbb6431f16d None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:36.069073 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a097ec9-c05e-4fb4-918f-4cbb6431f16d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:36.076749 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:36.079065 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a097ec9-c05e-4fb4-918f-4cbb6431f16d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:36.080498 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a097ec9-c05e-4fb4-918f-4cbb6431f16d None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:36.095385 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-1a097ec9-c05e-4fb4-918f-4cbb6431f16d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:23:36.095385 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:36.095385 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:36.095385 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:36.095385 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:36.095385 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:36.095385 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:36.095385 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:36.095385 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:36.095385 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:36.095385 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:36.095385 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:36.095385 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 08:23:36.095385 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 08:23:36.095385 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 08:23:36.095385 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 08:23:36.096385 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:36.096385 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:36.096385 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:36.096385 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:36.096385 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:36.096385 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:36.096385 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:36.096385 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:36.096385 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:36.096385 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:36.096385 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:36.096385 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:36.096385 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:36.096385 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:36.096385 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:36.096385 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:36.096385 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 08:23:36.097532 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 08:23:36.097532 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:23:36.097532 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:23:36.097532 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 259/504] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:23:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:36.097750 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40303c8-70e1-4f55-9612-0e45557070e0 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:36.097996 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40303c8-70e1-4f55-9612-0e45557070e0 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:36.099304 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40303c8-70e1-4f55-9612-0e45557070e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:36.117208 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-b40303c8-70e1-4f55-9612-0e45557070e0 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:23:36.118380 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-b40303c8-70e1-4f55-9612-0e45557070e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:23:36.149920 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f06b44-7eaf-4655-9031-d3c410c90952 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:36.183736 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a72f692-a2f4-45e8-bec5-f15a51f327e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:36.184288 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a72f692-a2f4-45e8-bec5-f15a51f327e9 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:36.184568 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a72f692-a2f4-45e8-bec5-f15a51f327e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:36.184838 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a72f692-a2f4-45e8-bec5-f15a51f327e9 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:36.186265 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 260/505] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:23:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 08:23:36.187084 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40303c8-70e1-4f55-9612-0e45557070e0 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:36.191802 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a72f692-a2f4-45e8-bec5-f15a51f327e9 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:36.193263 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a72f692-a2f4-45e8-bec5-f15a51f327e9 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:36.205883 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-3a72f692-a2f4-45e8-bec5-f15a51f327e9 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 10 08:23:36.205883 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:36.205883 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:36.205883 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:36.205883 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:36.205883 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:36.205883 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:36.205883 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:36.205883 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:36.205883 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:36.205883 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:36.205883 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:36.205883 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 08:23:36.205883 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 08:23:36.205883 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 08:23:36.205883 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 08:23:36.206820 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:36.206820 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:36.206820 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 08:23:36.206820 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 08:23:36.206820 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 08:23:36.206820 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 08:23:36.206820 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:36.206820 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:36.206820 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:36.206820 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:36.206820 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:36.206820 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:36.206820 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:36.206820 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:36.206820 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:36.206820 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:36.206820 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:36.206820 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:36.212715 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:36.212715 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:36.212715 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 08:23:36.212715 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 08:23:36.212715 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 08:23:36.212715 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 08:23:36.212715 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 08:23:36.212715 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 08:23:36.212715 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 10 08:23:36.212715 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:23:36.212715 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 246/506] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Aug 10 08:23:36 2022] GET /identity/v3/users/cinder => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:36.212715 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a72f692-a2f4-45e8-bec5-f15a51f327e9 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:36.244603 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:36.294476 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40303c8-70e1-4f55-9612-0e45557070e0 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:36.300386 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:36.308723 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a72f692-a2f4-45e8-bec5-f15a51f327e9 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:36.327732 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58620187-098f-4a41-b214-472cf8ef82c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:36.328364 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58620187-098f-4a41-b214-472cf8ef82c3 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:36.328641 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58620187-098f-4a41-b214-472cf8ef82c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:36.328938 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58620187-098f-4a41-b214-472cf8ef82c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:36.339913 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58620187-098f-4a41-b214-472cf8ef82c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:36.341403 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58620187-098f-4a41-b214-472cf8ef82c3 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:36.344083 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cd3075-05e2-4d6c-925a-235b52d9dca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:36.344682 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cd3075-05e2-4d6c-925a-235b52d9dca5 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:36.344994 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cd3075-05e2-4d6c-925a-235b52d9dca5 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:36.345274 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cd3075-05e2-4d6c-925a-235b52d9dca5 None admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:36.345910 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cd3075-05e2-4d6c-925a-235b52d9dca5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:36.346211 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cd3075-05e2-4d6c-925a-235b52d9dca5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:36.347551 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cd3075-05e2-4d6c-925a-235b52d9dca5 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:36.358782 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-58620187-098f-4a41-b214-472cf8ef82c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:23:36.358782 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:36.358782 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:36.358782 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:36.358782 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:36.358782 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:36.358782 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:36.358782 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:36.358782 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:36.358782 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:36.358782 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:36.358782 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:36.358782 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 08:23:36.358782 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 08:23:36.358782 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 08:23:36.358782 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 08:23:36.359694 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:36.359694 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:36.359694 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:36.359694 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:36.359694 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:36.359694 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:36.359694 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:36.359694 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:36.359694 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:36.359694 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:36.359694 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:36.359694 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:36.359694 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:36.359694 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:36.359694 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:36.359694 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:36.359694 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 08:23:36.360581 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 08:23:36.360581 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:23:36.360581 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:23:36.360581 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 261/507] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:23:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:36.360798 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 247/508] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Aug 10 08:23:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:36.361382 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58620187-098f-4a41-b214-472cf8ef82c3 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:36.362420 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cd3075-05e2-4d6c-925a-235b52d9dca5 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:36.464256 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cd3075-05e2-4d6c-925a-235b52d9dca5 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:36.467004 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:36.470070 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58620187-098f-4a41-b214-472cf8ef82c3 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:36.498334 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b227799-45a2-4899-9d15-d6f738cad293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:36.498960 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b227799-45a2-4899-9d15-d6f738cad293 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:36.499235 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b227799-45a2-4899-9d15-d6f738cad293 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:36.499514 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b227799-45a2-4899-9d15-d6f738cad293 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:36.504056 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01a0e4e-7897-40ab-9e55-59329e5f6acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:36.504849 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01a0e4e-7897-40ab-9e55-59329e5f6acb None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:36.505150 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01a0e4e-7897-40ab-9e55-59329e5f6acb None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:36.505420 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01a0e4e-7897-40ab-9e55-59329e5f6acb None admin] PATH_INFO: `/v3/domains` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:36.506049 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01a0e4e-7897-40ab-9e55-59329e5f6acb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:36.506348 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01a0e4e-7897-40ab-9e55-59329e5f6acb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:36.507706 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01a0e4e-7897-40ab-9e55-59329e5f6acb None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:36.509457 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b227799-45a2-4899-9d15-d6f738cad293 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 08:23:36.514092 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-0b227799-45a2-4899-9d15-d6f738cad293 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 08:23:36.514092 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:36.514092 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:36.514092 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:36.514092 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:36.514092 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:36.514092 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:36.514092 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:36.514092 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:36.514092 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:36.514092 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:36.514092 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:36.514092 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 08:23:36.514092 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 08:23:36.514092 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 08:23:36.514092 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 08:23:36.514994 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 08:23:36.514994 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 08:23:36.514994 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 08:23:36.514994 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 08:23:36.514994 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:36.514994 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:36.514994 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:36.514994 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:36.514994 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:36.514994 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:36.514994 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:36.514994 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:36.514994 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:36.514994 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:36.514994 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:36.514994 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:36.515981 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:36.515981 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:36.515981 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:36.515981 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:36.515981 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 08:23:36.515981 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 08:23:36.515981 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 08:23:36.515981 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 08:23:36.515981 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 08:23:36.515981 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 08:23:36.515981 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 08:23:36.515981 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:23:36.515981 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 248/509] 10.222.0.57 () {56 vars in 1102 bytes} [Wed Aug 10 08:23:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:36.515981 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b227799-45a2-4899-9d15-d6f738cad293 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:36.527186 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 262/510] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Aug 10 08:23:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:36.527186 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01a0e4e-7897-40ab-9e55-59329e5f6acb None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:36.530576 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:36.616106 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b227799-45a2-4899-9d15-d6f738cad293 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:36.623565 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01a0e4e-7897-40ab-9e55-59329e5f6acb None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:36.649836 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15bff3e-828b-4530-bf95-4b68dc9c19e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:36.650437 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15bff3e-828b-4530-bf95-4b68dc9c19e4 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:36.650716 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15bff3e-828b-4530-bf95-4b68dc9c19e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:36.650987 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15bff3e-828b-4530-bf95-4b68dc9c19e4 None admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:36.651712 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15bff3e-828b-4530-bf95-4b68dc9c19e4 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:36.652016 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15bff3e-828b-4530-bf95-4b68dc9c19e4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:36.653490 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15bff3e-828b-4530-bf95-4b68dc9c19e4 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:36.657530 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720dbe67-88ad-4db4-916f-8c3b2ff39e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:36.658149 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720dbe67-88ad-4db4-916f-8c3b2ff39e6f None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:36.658427 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720dbe67-88ad-4db4-916f-8c3b2ff39e6f None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:36.658697 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720dbe67-88ad-4db4-916f-8c3b2ff39e6f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:36.666225 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720dbe67-88ad-4db4-916f-8c3b2ff39e6f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:36.667660 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720dbe67-88ad-4db4-916f-8c3b2ff39e6f None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:36.670661 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 249/511] 10.222.0.57 () {56 vars in 1137 bytes} [Wed Aug 10 08:23:36 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:36.671460 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15bff3e-828b-4530-bf95-4b68dc9c19e4 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:36.681609 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-720dbe67-88ad-4db4-916f-8c3b2ff39e6f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 08:23:36.681609 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:36.681609 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:36.681609 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:36.681609 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:36.681609 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:36.681609 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:36.681609 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:36.681609 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:36.681609 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:36.681609 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:36.681609 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:36.681609 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 08:23:36.681609 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 08:23:36.681609 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 08:23:36.681609 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 08:23:36.682517 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:36.682517 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:36.682517 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 08:23:36.682517 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 08:23:36.682517 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 08:23:36.682517 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 08:23:36.682517 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:36.682517 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:36.682517 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:36.682517 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:36.682517 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:36.682517 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:36.682517 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:36.682517 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:36.682517 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:36.682517 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:36.682517 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:36.682517 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:36.683419 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:36.683419 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:36.683419 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 08:23:36.683419 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 08:23:36.683419 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 08:23:36.683419 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 08:23:36.683419 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 08:23:36.683419 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 08:23:36.683419 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 08:23:36.683419 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:23:36.683419 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 263/512] 10.222.0.57 () {56 vars in 1093 bytes} [Wed Aug 10 08:23:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:36.684001 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720dbe67-88ad-4db4-916f-8c3b2ff39e6f None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:36.704968 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:36.743487 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:36.773472 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15bff3e-828b-4530-bf95-4b68dc9c19e4 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:36.782132 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720dbe67-88ad-4db4-916f-8c3b2ff39e6f None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:36.808665 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf074ed-ed06-41bb-880a-a1c9fbf961d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:36.809303 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf074ed-ed06-41bb-880a-a1c9fbf961d0 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:36.809584 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf074ed-ed06-41bb-880a-a1c9fbf961d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:36.809865 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf074ed-ed06-41bb-880a-a1c9fbf961d0 None admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:36.810501 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf074ed-ed06-41bb-880a-a1c9fbf961d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:36.810804 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf074ed-ed06-41bb-880a-a1c9fbf961d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:36.812164 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf074ed-ed06-41bb-880a-a1c9fbf961d0 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:36.814634 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b91ef21-dffa-45d5-8feb-f15bb599809e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:36.815230 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b91ef21-dffa-45d5-8feb-f15bb599809e None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:36.815508 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b91ef21-dffa-45d5-8feb-f15bb599809e None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:36.815778 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b91ef21-dffa-45d5-8feb-f15bb599809e None admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:36.816426 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b91ef21-dffa-45d5-8feb-f15bb599809e None admin] RBAC: Adding query filter params (name=service) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:36.816858 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b91ef21-dffa-45d5-8feb-f15bb599809e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:36.818381 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b91ef21-dffa-45d5-8feb-f15bb599809e None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:36.823222 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 250/513] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Aug 10 08:23:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:36.824067 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf074ed-ed06-41bb-880a-a1c9fbf961d0 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:36.830834 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 264/514] 10.222.0.57 () {56 vars in 1103 bytes} [Wed Aug 10 08:23:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:36.830834 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b91ef21-dffa-45d5-8feb-f15bb599809e None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:36.907760 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:36.925546 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b91ef21-dffa-45d5-8feb-f15bb599809e None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:36.931706 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf074ed-ed06-41bb-880a-a1c9fbf961d0 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:36.953987 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:36.959194 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76f0110-51d9-460f-9209-91ee8be6f055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:36.959788 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76f0110-51d9-460f-9209-91ee8be6f055 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:36.960064 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76f0110-51d9-460f-9209-91ee8be6f055 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:36.960208 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d827599a-2e64-459f-b816-a255a88f0ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:36.960334 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76f0110-51d9-460f-9209-91ee8be6f055 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:36.960823 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d827599a-2e64-459f-b816-a255a88f0ffc None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:36.961163 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d827599a-2e64-459f-b816-a255a88f0ffc None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:36.961405 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d827599a-2e64-459f-b816-a255a88f0ffc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:36.967809 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d827599a-2e64-459f-b816-a255a88f0ffc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:36.969276 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d827599a-2e64-459f-b816-a255a88f0ffc None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:36.980860 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-d827599a-2e64-459f-b816-a255a88f0ffc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 08:23:36.980860 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:36.980860 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:36.980860 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:36.980860 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:36.980860 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:36.980860 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:36.980860 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:36.980860 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:36.980860 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:36.980860 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:36.980860 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:36.980860 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 08:23:36.980860 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 08:23:36.980860 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 08:23:36.981959 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 08:23:36.981959 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:36.981959 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:36.981959 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:36.981959 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:36.981959 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:36.981959 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:36.981959 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:36.981959 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:36.981959 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:36.981959 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:36.981959 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:36.981959 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:36.981959 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:36.981959 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:36.981959 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:36.981959 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:36.982874 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 08:23:36.982874 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 08:23:36.982874 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 08:23:36.982874 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 08:23:36.982874 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 08:23:36.982874 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 08:23:36.982874 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 08:23:36.982874 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:23:36.982874 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 251/515] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Aug 10 08:23:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:36.983353 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d827599a-2e64-459f-b816-a255a88f0ffc None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:37.030019 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76f0110-51d9-460f-9209-91ee8be6f055 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:37.031410 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76f0110-51d9-460f-9209-91ee8be6f055 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:37.086821 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d827599a-2e64-459f-b816-a255a88f0ffc None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:37.115439 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:37.123643 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3a1d11-63f4-419d-974d-96a58a6a7d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:37.124246 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3a1d11-63f4-419d-974d-96a58a6a7d4c None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:37.124525 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3a1d11-63f4-419d-974d-96a58a6a7d4c None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:37.128781 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3a1d11-63f4-419d-974d-96a58a6a7d4c None admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:37.128781 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3a1d11-63f4-419d-974d-96a58a6a7d4c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:37.128781 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3a1d11-63f4-419d-974d-96a58a6a7d4c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:37.128781 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3a1d11-63f4-419d-974d-96a58a6a7d4c None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:37.135234 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 265/516] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Aug 10 08:23:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 08:23:37.136032 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76f0110-51d9-460f-9209-91ee8be6f055 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:37.150475 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 252/517] 10.222.0.57 () {56 vars in 1133 bytes} [Wed Aug 10 08:23:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:37.151320 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3a1d11-63f4-419d-974d-96a58a6a7d4c None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:37.202813 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:37.241206 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76f0110-51d9-460f-9209-91ee8be6f055 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:37.255774 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3a1d11-63f4-419d-974d-96a58a6a7d4c None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:37.279320 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f933e21c-c6a1-4d8b-a106-e450db25a46e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:37.280560 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f933e21c-c6a1-4d8b-a106-e450db25a46e None admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:37.280723 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f933e21c-c6a1-4d8b-a106-e450db25a46e None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:37.281193 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f933e21c-c6a1-4d8b-a106-e450db25a46e None admin] PATH_INFO: `/v3/users` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:37.282226 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f933e21c-c6a1-4d8b-a106-e450db25a46e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:37.284775 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f933e21c-c6a1-4d8b-a106-e450db25a46e None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:37.291531 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c47145d-a846-47a4-9522-4ba012ac15fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:37.292149 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c47145d-a846-47a4-9522-4ba012ac15fa None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:37.292425 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c47145d-a846-47a4-9522-4ba012ac15fa None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:37.292693 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c47145d-a846-47a4-9522-4ba012ac15fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:37.303549 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c47145d-a846-47a4-9522-4ba012ac15fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:37.304997 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c47145d-a846-47a4-9522-4ba012ac15fa None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:37.320235 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-6c47145d-a846-47a4-9522-4ba012ac15fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:23:37.320235 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:37.320235 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:37.320235 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:37.320235 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:37.320235 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:37.320235 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:37.320235 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:37.320235 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:37.320235 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:37.320235 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:37.320235 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:37.320235 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 08:23:37.320235 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 08:23:37.320235 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 08:23:37.320235 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 08:23:37.321354 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:37.321354 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:37.321354 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:37.321354 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:37.321354 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:37.321354 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:37.321354 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:37.321354 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:37.321354 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:37.321354 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:37.321354 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:37.321354 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:37.321354 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:37.321354 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:37.321354 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:37.321354 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:37.321354 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 08:23:37.322209 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 08:23:37.322209 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:23:37.322209 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:23:37.322209 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 253/518] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:23:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:37.322209 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c47145d-a846-47a4-9522-4ba012ac15fa None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:37.367368 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 266/519] 10.222.0.57 () {60 vars in 1132 bytes} [Wed Aug 10 08:23:37 2022] POST /identity/v3/users => generated 247 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:23:37.368204 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f933e21c-c6a1-4d8b-a106-e450db25a46e None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:37.419835 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:37.451779 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:37.457288 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c47145d-a846-47a4-9522-4ba012ac15fa None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:37.490038 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f933e21c-c6a1-4d8b-a106-e450db25a46e None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:37.507175 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5336389c-2d05-45ed-927f-800635f62f8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:37.508033 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5336389c-2d05-45ed-927f-800635f62f8b None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:37.508475 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5336389c-2d05-45ed-927f-800635f62f8b None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:37.508864 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5336389c-2d05-45ed-927f-800635f62f8b None admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:37.509935 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5336389c-2d05-45ed-927f-800635f62f8b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:37.510377 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5336389c-2d05-45ed-927f-800635f62f8b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:37.512556 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5336389c-2d05-45ed-927f-800635f62f8b None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:37.529802 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009f85ab-14bf-4f28-8aeb-2c81736ea903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:37.530791 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009f85ab-14bf-4f28-8aeb-2c81736ea903 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:37.531242 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009f85ab-14bf-4f28-8aeb-2c81736ea903 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:37.531706 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009f85ab-14bf-4f28-8aeb-2c81736ea903 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:37.540843 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 254/520] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Aug 10 08:23:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:37.542396 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5336389c-2d05-45ed-927f-800635f62f8b None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:37.547666 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009f85ab-14bf-4f28-8aeb-2c81736ea903 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:37.549242 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009f85ab-14bf-4f28-8aeb-2c81736ea903 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:37.565309 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-009f85ab-14bf-4f28-8aeb-2c81736ea903 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:23:37.565309 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:37.565309 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:37.565309 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:37.565309 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:37.565309 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:37.565309 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:37.565309 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:37.565309 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:37.565309 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:37.565309 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:37.565309 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:37.565309 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 08:23:37.565309 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 08:23:37.565309 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 08:23:37.565309 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 08:23:37.566478 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:37.566478 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:37.566478 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:37.566478 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:37.566478 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:37.566478 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:37.566478 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:37.566478 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:37.566478 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:37.566478 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:37.566478 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:37.566478 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:37.566478 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:37.566478 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:37.566478 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:37.566478 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:37.566478 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 08:23:37.567957 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 08:23:37.567957 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:23:37.567957 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:23:37.569638 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 267/521] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:23:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:37.570589 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009f85ab-14bf-4f28-8aeb-2c81736ea903 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:37.673063 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:37.681068 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5336389c-2d05-45ed-927f-800635f62f8b None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:37.685577 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009f85ab-14bf-4f28-8aeb-2c81736ea903 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:37.723516 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0173a0a-82c0-4dae-ad12-3cbc7a6971f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:37.724349 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0173a0a-82c0-4dae-ad12-3cbc7a6971f0 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:37.724737 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0173a0a-82c0-4dae-ad12-3cbc7a6971f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:37.725158 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0173a0a-82c0-4dae-ad12-3cbc7a6971f0 None admin] PATH_INFO: `/v3/domains` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:37.726065 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0173a0a-82c0-4dae-ad12-3cbc7a6971f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:37.726496 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0173a0a-82c0-4dae-ad12-3cbc7a6971f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:37.727340 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8731306-eeff-4195-a664-436e035b2ffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:37.727946 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8731306-eeff-4195-a664-436e035b2ffe None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:37.728224 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8731306-eeff-4195-a664-436e035b2ffe None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:37.728490 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0173a0a-82c0-4dae-ad12-3cbc7a6971f0 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:37.728585 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8731306-eeff-4195-a664-436e035b2ffe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:37.750514 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 268/522] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Aug 10 08:23:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:37.750514 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0173a0a-82c0-4dae-ad12-3cbc7a6971f0 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:37.764402 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:37.810704 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8731306-eeff-4195-a664-436e035b2ffe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:37.812905 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8731306-eeff-4195-a664-436e035b2ffe None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:37.875841 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0173a0a-82c0-4dae-ad12-3cbc7a6971f0 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:37.915068 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e45415-b78f-40e1-a058-13bb5d25efee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:37.915674 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e45415-b78f-40e1-a058-13bb5d25efee None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:37.915948 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e45415-b78f-40e1-a058-13bb5d25efee None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:37.916212 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e45415-b78f-40e1-a058-13bb5d25efee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:37.916951 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e45415-b78f-40e1-a058-13bb5d25efee None admin] RBAC: Adding query filter params (role.id=73fdd60e2f804ee69b76fa9145a472ff, scope.project.id=b49e381419e940278bcb6111899589b5, user.id=3263af99fc2f48e79fd1e5b8a1e555a4) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:37.917260 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e45415-b78f-40e1-a058-13bb5d25efee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:37.918674 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e45415-b78f-40e1-a058-13bb5d25efee None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:37.933100 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 269/523] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Aug 10 08:23:37 2022] GET /identity/v3/role_assignments?user.id=3263af99fc2f48e79fd1e5b8a1e555a4&scope.project.id=b49e381419e940278bcb6111899589b5&role.id=73fdd60e2f804ee69b76fa9145a472ff => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:37.933100 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e45415-b78f-40e1-a058-13bb5d25efee None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:37.933522 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 255/524] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Aug 10 08:23:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 08:23:37.933522 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8731306-eeff-4195-a664-436e035b2ffe None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:37.956028 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:38.003422 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:38.060100 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8731306-eeff-4195-a664-436e035b2ffe None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:38.077490 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e45415-b78f-40e1-a058-13bb5d25efee None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:38.101257 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc16d5f-bb50-48eb-9d01-992c166f1d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:38.102625 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc16d5f-bb50-48eb-9d01-992c166f1d21 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:38.103132 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc16d5f-bb50-48eb-9d01-992c166f1d21 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:38.103603 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc16d5f-bb50-48eb-9d01-992c166f1d21 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:38.112813 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc16d5f-bb50-48eb-9d01-992c166f1d21 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:38.115207 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc16d5f-bb50-48eb-9d01-992c166f1d21 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:38.117900 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26137a43-59a0-43a9-ab89-bd3ded43d457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:38.118809 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26137a43-59a0-43a9-ab89-bd3ded43d457 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:38.119248 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26137a43-59a0-43a9-ab89-bd3ded43d457 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:38.119729 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26137a43-59a0-43a9-ab89-bd3ded43d457 None admin] PATH_INFO: `/v3/domains` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:38.121520 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26137a43-59a0-43a9-ab89-bd3ded43d457 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:38.121721 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26137a43-59a0-43a9-ab89-bd3ded43d457 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:38.123729 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26137a43-59a0-43a9-ab89-bd3ded43d457 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:38.143062 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-9cc16d5f-bb50-48eb-9d01-992c166f1d21 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 10 08:23:38.143062 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:38.143062 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:38.143062 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:38.143062 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:38.143062 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:38.143062 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:38.143062 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:38.143062 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:38.143062 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:38.143062 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:38.143062 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:38.143062 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 08:23:38.143062 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 08:23:38.143062 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 08:23:38.143062 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 08:23:38.144427 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:38.144427 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:38.144427 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 08:23:38.144427 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 08:23:38.144427 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 08:23:38.144427 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 08:23:38.144427 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:38.144427 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:38.144427 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:38.144427 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:38.144427 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:38.144427 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:38.144427 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:38.144427 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:38.144427 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:38.144427 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:38.144427 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:38.144427 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:38.146647 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:38.146647 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:38.146647 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 08:23:38.146647 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 08:23:38.146647 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 08:23:38.146647 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 08:23:38.146647 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 08:23:38.146647 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 08:23:38.146647 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 10 08:23:38.146647 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:23:38.147404 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 270/525] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Aug 10 08:23:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:38.147404 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26137a43-59a0-43a9-ab89-bd3ded43d457 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:38.149800 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 256/526] 10.222.0.57 () {56 vars in 1102 bytes} [Wed Aug 10 08:23:37 2022] GET /identity/v3/users/neutron => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:38.154018 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc16d5f-bb50-48eb-9d01-992c166f1d21 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:38.238073 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:38.265805 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26137a43-59a0-43a9-ab89-bd3ded43d457 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:38.270093 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc16d5f-bb50-48eb-9d01-992c166f1d21 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:38.292643 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:38.303432 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caa5d59-925a-4d93-b5d5-8a1f2cc59824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:38.309027 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caa5d59-925a-4d93-b5d5-8a1f2cc59824 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:38.309027 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caa5d59-925a-4d93-b5d5-8a1f2cc59824 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:38.309027 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caa5d59-925a-4d93-b5d5-8a1f2cc59824 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:38.309300 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500e6262-ab5f-4c52-b0f4-8203dafc8e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:38.309682 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e6262-ab5f-4c52-b0f4-8203dafc8e0e None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:38.309963 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e6262-ab5f-4c52-b0f4-8203dafc8e0e None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:38.310239 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e6262-ab5f-4c52-b0f4-8203dafc8e0e None admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:38.310975 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500e6262-ab5f-4c52-b0f4-8203dafc8e0e None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:38.311282 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500e6262-ab5f-4c52-b0f4-8203dafc8e0e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:38.312710 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500e6262-ab5f-4c52-b0f4-8203dafc8e0e None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:38.318238 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caa5d59-925a-4d93-b5d5-8a1f2cc59824 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:38.319867 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caa5d59-925a-4d93-b5d5-8a1f2cc59824 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:38.336161 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-8caa5d59-925a-4d93-b5d5-8a1f2cc59824 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:23:38.336161 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:38.336161 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:38.336161 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:38.336161 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:38.336161 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:38.336161 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:38.336161 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:38.336161 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:38.336161 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:38.336161 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:38.336161 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:38.336161 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 08:23:38.336161 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 08:23:38.336161 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 08:23:38.336161 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 08:23:38.337437 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:38.337437 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:38.337437 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:38.337437 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:38.337437 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:38.337437 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:38.337437 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:38.337437 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:38.337437 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:38.337437 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:38.337437 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:38.337437 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:38.337437 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:38.337437 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:38.337437 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:38.337437 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:38.337437 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 08:23:38.338570 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 08:23:38.338570 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:23:38.338570 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:23:38.338737 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 257/527] 10.222.0.57 () {56 vars in 1139 bytes} [Wed Aug 10 08:23:38 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:38.339100 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500e6262-ab5f-4c52-b0f4-8203dafc8e0e None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:38.340516 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 271/528] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:23:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:38.341763 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caa5d59-925a-4d93-b5d5-8a1f2cc59824 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:38.453816 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500e6262-ab5f-4c52-b0f4-8203dafc8e0e None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:38.462169 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caa5d59-925a-4d93-b5d5-8a1f2cc59824 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:38.490414 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:38.492717 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5e8461-8e52-4956-b774-82bebd1ae221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:38.493379 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5e8461-8e52-4956-b774-82bebd1ae221 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:38.493660 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5e8461-8e52-4956-b774-82bebd1ae221 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:38.493932 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5e8461-8e52-4956-b774-82bebd1ae221 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:38.500843 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16b4307-db40-43c2-a4b3-5c9ffd595bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:38.504710 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16b4307-db40-43c2-a4b3-5c9ffd595bb0 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:38.504811 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5e8461-8e52-4956-b774-82bebd1ae221 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:38.504889 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16b4307-db40-43c2-a4b3-5c9ffd595bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:38.504889 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16b4307-db40-43c2-a4b3-5c9ffd595bb0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:38.505169 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5e8461-8e52-4956-b774-82bebd1ae221 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:38.516739 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-ac5e8461-8e52-4956-b774-82bebd1ae221 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 08:23:38.516739 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:38.516739 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:38.516739 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:38.516739 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:38.516739 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:38.516739 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:38.516739 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:38.516739 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:38.516739 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:38.516739 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:38.516739 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:38.516739 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 08:23:38.516739 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 08:23:38.516739 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 08:23:38.518358 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 08:23:38.518358 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:38.518358 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:38.518358 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:38.518358 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:38.518358 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:38.518358 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:38.518358 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:38.518358 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:38.518358 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:38.518358 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:38.518358 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:38.518358 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:38.518358 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:38.518358 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:38.518358 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:38.518358 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:38.519410 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 08:23:38.519410 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 08:23:38.519410 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 08:23:38.519410 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 08:23:38.519410 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 08:23:38.519410 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 08:23:38.519410 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 08:23:38.519410 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:23:38.519410 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 258/529] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Aug 10 08:23:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:38.519410 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5e8461-8e52-4956-b774-82bebd1ae221 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:38.569941 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:38.594561 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16b4307-db40-43c2-a4b3-5c9ffd595bb0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:38.597030 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16b4307-db40-43c2-a4b3-5c9ffd595bb0 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:38.638149 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5e8461-8e52-4956-b774-82bebd1ae221 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:38.675984 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215567fa-3ae8-4f6f-b87c-5b5fd4d19961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:38.676594 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215567fa-3ae8-4f6f-b87c-5b5fd4d19961 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:38.676870 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215567fa-3ae8-4f6f-b87c-5b5fd4d19961 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:38.677235 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215567fa-3ae8-4f6f-b87c-5b5fd4d19961 None admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:38.678008 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215567fa-3ae8-4f6f-b87c-5b5fd4d19961 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:38.678314 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215567fa-3ae8-4f6f-b87c-5b5fd4d19961 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:38.679707 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215567fa-3ae8-4f6f-b87c-5b5fd4d19961 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:38.693055 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 259/530] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Aug 10 08:23:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:38.694023 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215567fa-3ae8-4f6f-b87c-5b5fd4d19961 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:38.728564 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 272/531] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Aug 10 08:23:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 08:23:38.729805 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16b4307-db40-43c2-a4b3-5c9ffd595bb0 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:38.762651 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:38.808635 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:38.812527 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215567fa-3ae8-4f6f-b87c-5b5fd4d19961 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:38.850854 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe47de69-aa07-47f2-9048-dec3b50ed3e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:38.851705 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe47de69-aa07-47f2-9048-dec3b50ed3e3 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:38.852152 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe47de69-aa07-47f2-9048-dec3b50ed3e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:38.852568 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe47de69-aa07-47f2-9048-dec3b50ed3e3 None admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:38.853548 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe47de69-aa07-47f2-9048-dec3b50ed3e3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:38.854054 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe47de69-aa07-47f2-9048-dec3b50ed3e3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:38.855650 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe47de69-aa07-47f2-9048-dec3b50ed3e3 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:38.856691 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16b4307-db40-43c2-a4b3-5c9ffd595bb0 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:38.870649 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 260/532] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Aug 10 08:23:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:38.871796 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe47de69-aa07-47f2-9048-dec3b50ed3e3 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:38.902226 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2936f6b-9add-4105-9292-d89b62b16059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:38.903012 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2936f6b-9add-4105-9292-d89b62b16059 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:38.903698 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2936f6b-9add-4105-9292-d89b62b16059 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:38.904068 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2936f6b-9add-4105-9292-d89b62b16059 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:38.913643 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2936f6b-9add-4105-9292-d89b62b16059 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:38.915238 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2936f6b-9add-4105-9292-d89b62b16059 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:38.933839 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-b2936f6b-9add-4105-9292-d89b62b16059 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 08:23:38.933839 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:38.933839 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:38.933839 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:38.933839 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:38.933839 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:38.933839 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:38.933839 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:38.933839 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:38.933839 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:38.933839 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:38.933839 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:38.933839 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 08:23:38.933839 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 08:23:38.933839 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 08:23:38.934919 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 08:23:38.934919 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:38.934919 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:38.934919 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:38.934919 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:38.934919 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:38.934919 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:38.934919 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:38.934919 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:38.934919 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:38.934919 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:38.934919 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:38.934919 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:38.934919 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:38.934919 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:38.934919 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:38.934919 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:38.935981 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 08:23:38.935981 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 08:23:38.935981 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 08:23:38.935981 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 08:23:38.935981 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 08:23:38.935981 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 08:23:38.935981 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 08:23:38.935981 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:23:38.938148 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 273/533] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Aug 10 08:23:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:38.939114 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2936f6b-9add-4105-9292-d89b62b16059 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:39.007867 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe47de69-aa07-47f2-9048-dec3b50ed3e3 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:39.040939 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:39.051861 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662d51c6-4a3e-4012-9258-3ea62edc5ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:39.054721 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662d51c6-4a3e-4012-9258-3ea62edc5ba1 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:39.054721 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662d51c6-4a3e-4012-9258-3ea62edc5ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:39.054721 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662d51c6-4a3e-4012-9258-3ea62edc5ba1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:39.065811 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2936f6b-9add-4105-9292-d89b62b16059 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:39.067670 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662d51c6-4a3e-4012-9258-3ea62edc5ba1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:39.071074 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662d51c6-4a3e-4012-9258-3ea62edc5ba1 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:39.099891 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-662d51c6-4a3e-4012-9258-3ea62edc5ba1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:23:39.099891 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:39.099891 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:39.099891 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:39.099891 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:39.099891 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:39.099891 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:39.099891 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:39.099891 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:39.099891 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:39.099891 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:39.099891 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:39.099891 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 08:23:39.099891 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 08:23:39.099891 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 08:23:39.099891 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 08:23:39.100968 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:39.100968 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:39.100968 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:39.100968 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:39.100968 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:39.100968 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:39.100968 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:39.100968 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:39.100968 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:39.100968 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:39.100968 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:39.100968 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:39.100968 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:39.100968 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:39.100968 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:39.100968 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:39.100968 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 08:23:39.102278 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 08:23:39.102278 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:23:39.102278 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:23:39.102278 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 261/534] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:23:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:39.102278 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662d51c6-4a3e-4012-9258-3ea62edc5ba1 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:39.115044 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6269754-fa4b-4a2d-ba29-261e676fd25c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:39.115660 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6269754-fa4b-4a2d-ba29-261e676fd25c None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:39.115940 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6269754-fa4b-4a2d-ba29-261e676fd25c None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:39.116209 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6269754-fa4b-4a2d-ba29-261e676fd25c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:39.116962 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6269754-fa4b-4a2d-ba29-261e676fd25c None admin] RBAC: Adding query filter params (role.id=73fdd60e2f804ee69b76fa9145a472ff, scope.project.id=b49e381419e940278bcb6111899589b5, user.id=f1ba74e28ced4cbba857f54222dfbe41) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:39.117283 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6269754-fa4b-4a2d-ba29-261e676fd25c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:39.118774 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6269754-fa4b-4a2d-ba29-261e676fd25c None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:39.125031 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:39.132945 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 274/535] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Aug 10 08:23:38 2022] GET /identity/v3/role_assignments?user.id=f1ba74e28ced4cbba857f54222dfbe41&scope.project.id=b49e381419e940278bcb6111899589b5&role.id=73fdd60e2f804ee69b76fa9145a472ff => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:39.132945 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6269754-fa4b-4a2d-ba29-261e676fd25c None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:39.245426 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6269754-fa4b-4a2d-ba29-261e676fd25c None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:39.280292 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662d51c6-4a3e-4012-9258-3ea62edc5ba1 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:39.330898 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7c16e7-06ec-4cc3-9f41-1a4679eb5b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:39.331902 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7c16e7-06ec-4cc3-9f41-1a4679eb5b12 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:39.333643 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7c16e7-06ec-4cc3-9f41-1a4679eb5b12 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:39.334127 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7c16e7-06ec-4cc3-9f41-1a4679eb5b12 None admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:39.335037 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7c16e7-06ec-4cc3-9f41-1a4679eb5b12 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:39.335644 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7c16e7-06ec-4cc3-9f41-1a4679eb5b12 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:39.337405 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7c16e7-06ec-4cc3-9f41-1a4679eb5b12 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:39.349944 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec4c5e8-100d-4c04-ac89-ee8c747a21a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:39.350811 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec4c5e8-100d-4c04-ac89-ee8c747a21a3 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:39.351260 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec4c5e8-100d-4c04-ac89-ee8c747a21a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:39.351583 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec4c5e8-100d-4c04-ac89-ee8c747a21a3 None admin] PATH_INFO: `/v3/domains` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:39.352520 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec4c5e8-100d-4c04-ac89-ee8c747a21a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:39.353604 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec4c5e8-100d-4c04-ac89-ee8c747a21a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:39.355069 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec4c5e8-100d-4c04-ac89-ee8c747a21a3 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:39.357579 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 262/536] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Aug 10 08:23:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:39.364577 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7c16e7-06ec-4cc3-9f41-1a4679eb5b12 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:39.383212 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 275/537] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Aug 10 08:23:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:39.392112 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec4c5e8-100d-4c04-ac89-ee8c747a21a3 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:39.414880 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:39.452859 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:39.491465 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7c16e7-06ec-4cc3-9f41-1a4679eb5b12 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:39.517482 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec4c5e8-100d-4c04-ac89-ee8c747a21a3 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:39.531197 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641c8287-053b-482d-be31-f474aee42b7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:39.532204 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641c8287-053b-482d-be31-f474aee42b7e None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:39.532702 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641c8287-053b-482d-be31-f474aee42b7e None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:39.533196 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641c8287-053b-482d-be31-f474aee42b7e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:39.534282 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641c8287-053b-482d-be31-f474aee42b7e None admin] RBAC: Adding query filter params (role.id=73fdd60e2f804ee69b76fa9145a472ff, scope.project.id=b49e381419e940278bcb6111899589b5, user.id=a943008193914f76a663d489bd73e873) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:39.534909 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641c8287-053b-482d-be31-f474aee42b7e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:39.537390 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641c8287-053b-482d-be31-f474aee42b7e None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:39.549952 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 263/538] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Aug 10 08:23:39 2022] GET /identity/v3/role_assignments?user.id=a943008193914f76a663d489bd73e873&scope.project.id=b49e381419e940278bcb6111899589b5&role.id=73fdd60e2f804ee69b76fa9145a472ff => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:23:39.558081 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c604da-ac0d-449b-ae5f-dd560dd5ec15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:39.558735 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c604da-ac0d-449b-ae5f-dd560dd5ec15 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:39.559014 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c604da-ac0d-449b-ae5f-dd560dd5ec15 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:39.559285 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c604da-ac0d-449b-ae5f-dd560dd5ec15 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:39.567836 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c604da-ac0d-449b-ae5f-dd560dd5ec15 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:39.568342 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cfa97e-d974-404e-bb8f-dd345f83e1f0 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:39.568628 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cfa97e-d974-404e-bb8f-dd345f83e1f0 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:39.568898 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cfa97e-d974-404e-bb8f-dd345f83e1f0 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:39.569327 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c604da-ac0d-449b-ae5f-dd560dd5ec15 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:39.570519 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 264/539] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:23:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 08:23:39.575749 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65373204-9637-4954-a1b8-5ede494b2681 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:39.576053 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65373204-9637-4954-a1b8-5ede494b2681 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:39.576291 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65373204-9637-4954-a1b8-5ede494b2681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:39.581822 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-37c604da-ac0d-449b-ae5f-dd560dd5ec15 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 08:23:39.581822 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:39.581822 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:39.581822 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:39.581822 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:39.581822 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:39.581822 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:39.581822 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:39.581822 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:39.581822 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:39.581822 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:39.581822 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:39.581822 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 08:23:39.581822 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 08:23:39.581822 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 08:23:39.582737 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 08:23:39.582737 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:39.582737 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:39.582737 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:39.582737 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:39.582737 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:39.582737 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:39.582737 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:39.582737 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:39.582737 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:39.582737 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:39.582737 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:39.582737 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:39.582737 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:39.582737 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:39.582737 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:39.582737 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:39.583735 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 08:23:39.583735 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 08:23:39.583735 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 08:23:39.583735 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 08:23:39.583735 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 08:23:39.583735 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 08:23:39.583735 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 08:23:39.583735 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:23:39.583735 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 276/540] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Aug 10 08:23:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:39.588587 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c604da-ac0d-449b-ae5f-dd560dd5ec15 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:39.597257 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-65373204-9637-4954-a1b8-5ede494b2681 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:23:39.598971 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-65373204-9637-4954-a1b8-5ede494b2681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:23:39.666641 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:39.693517 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 265/541] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:23:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 08:23:39.721461 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65373204-9637-4954-a1b8-5ede494b2681 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:39.723711 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c604da-ac0d-449b-ae5f-dd560dd5ec15 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:39.766399 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a6b9fc-4d76-42ba-b417-b3183571f6d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:39.767312 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a6b9fc-4d76-42ba-b417-b3183571f6d1 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:39.767736 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a6b9fc-4d76-42ba-b417-b3183571f6d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:39.768166 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a6b9fc-4d76-42ba-b417-b3183571f6d1 None admin] PATH_INFO: `/v3/projects` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:39.769048 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a6b9fc-4d76-42ba-b417-b3183571f6d1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:39.769498 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a6b9fc-4d76-42ba-b417-b3183571f6d1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:39.771267 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a6b9fc-4d76-42ba-b417-b3183571f6d1 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:39.792275 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 277/542] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Aug 10 08:23:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:23:39.798172 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a6b9fc-4d76-42ba-b417-b3183571f6d1 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:39.843893 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:39.884743 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65373204-9637-4954-a1b8-5ede494b2681 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:39.922815 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeddbec-d14c-4712-a878-35530a007c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:39.923362 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeddbec-d14c-4712-a878-35530a007c7b None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:39.923632 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeddbec-d14c-4712-a878-35530a007c7b None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:39.923901 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeddbec-d14c-4712-a878-35530a007c7b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:39.945053 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceeddbec-d14c-4712-a878-35530a007c7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 08:23:39.949815 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-ceeddbec-d14c-4712-a878-35530a007c7b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 08:23:39.949815 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:39.949815 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:39.949815 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:39.949815 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:39.949815 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:39.949815 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:39.949815 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:39.949815 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:39.949815 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:39.949815 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:39.949815 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:39.949815 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 08:23:39.949815 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 08:23:39.949815 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 08:23:39.949815 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 08:23:39.950801 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 08:23:39.950801 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 08:23:39.950801 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 08:23:39.950801 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 08:23:39.950801 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:39.950801 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:39.950801 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:39.950801 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:39.950801 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:39.950801 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:39.950801 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:39.950801 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:39.950801 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:39.950801 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:39.950801 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:39.950801 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:39.951847 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:39.951847 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:39.951847 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:39.951847 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:39.951847 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 08:23:39.951847 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 08:23:39.951847 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 08:23:39.951847 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 08:23:39.951847 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 08:23:39.951847 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 08:23:39.951847 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 08:23:39.951847 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:23:39.951847 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 266/543] 10.222.0.57 () {56 vars in 1102 bytes} [Wed Aug 10 08:23:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 08:23:39.960113 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeddbec-d14c-4712-a878-35530a007c7b None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:39.977866 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a6b9fc-4d76-42ba-b417-b3183571f6d1 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:40.013884 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:40.015741 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf80a13-7e25-4649-9d52-947f012798f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:40.016345 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf80a13-7e25-4649-9d52-947f012798f8 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:40.016627 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf80a13-7e25-4649-9d52-947f012798f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:40.016896 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf80a13-7e25-4649-9d52-947f012798f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:40.017802 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf80a13-7e25-4649-9d52-947f012798f8 None admin] RBAC: Adding query filter params (role.id=73fdd60e2f804ee69b76fa9145a472ff, scope.project.id=b49e381419e940278bcb6111899589b5, user.id=d4863ecff21e408c9deca645b88d3727) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:40.018112 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf80a13-7e25-4649-9d52-947f012798f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:40.019579 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf80a13-7e25-4649-9d52-947f012798f8 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:40.026762 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 278/544] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Aug 10 08:23:39 2022] GET /identity/v3/role_assignments?user.id=d4863ecff21e408c9deca645b88d3727&scope.project.id=b49e381419e940278bcb6111899589b5&role.id=73fdd60e2f804ee69b76fa9145a472ff => generated 259 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:40.120491 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeddbec-d14c-4712-a878-35530a007c7b None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:40.148457 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:40.160055 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea7b020-807e-4c87-b3af-b315aba32f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:40.160775 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea7b020-807e-4c87-b3af-b315aba32f91 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:40.161142 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea7b020-807e-4c87-b3af-b315aba32f91 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:40.161429 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea7b020-807e-4c87-b3af-b315aba32f91 None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:40.162103 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea7b020-807e-4c87-b3af-b315aba32f91 None admin] RBAC: Adding query filter params (name=service) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:40.162404 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea7b020-807e-4c87-b3af-b315aba32f91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:40.163776 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea7b020-807e-4c87-b3af-b315aba32f91 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:40.172784 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 267/545] 10.222.0.57 () {56 vars in 1103 bytes} [Wed Aug 10 08:23:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:23:40.184071 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf80a13-7e25-4649-9d52-947f012798f8 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:40.303775 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf80a13-7e25-4649-9d52-947f012798f8 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:40.348135 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d66057a-6083-483c-a95b-2247150d6224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:40.349277 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d66057a-6083-483c-a95b-2247150d6224 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:40.349776 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d66057a-6083-483c-a95b-2247150d6224 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:40.350271 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d66057a-6083-483c-a95b-2247150d6224 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:40.366234 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d66057a-6083-483c-a95b-2247150d6224 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:40.368077 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d66057a-6083-483c-a95b-2247150d6224 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:40.388355 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-9d66057a-6083-483c-a95b-2247150d6224 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:23:40.388355 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:40.388355 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:40.388355 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:40.388355 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:40.388355 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:40.388355 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:40.388355 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:40.388355 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:40.388355 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:40.388355 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:40.388355 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:40.388355 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 08:23:40.388355 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 08:23:40.388355 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 08:23:40.388355 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 08:23:40.389368 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:40.389368 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:40.389368 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:40.389368 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:40.389368 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:40.389368 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:40.389368 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:40.389368 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:40.389368 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:40.389368 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:40.389368 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:40.389368 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:40.389368 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:40.389368 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:40.389368 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:40.389368 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:40.389368 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 08:23:40.390254 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 08:23:40.390254 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:23:40.390254 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:23:40.393289 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 279/546] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:23:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 08:23:40.400906 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea7b020-807e-4c87-b3af-b315aba32f91 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:40.588208 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea7b020-807e-4c87-b3af-b315aba32f91 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:40.626467 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fc5720-3612-4431-9af7-cbcdc928dc0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:40.627440 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fc5720-3612-4431-9af7-cbcdc928dc0b None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:40.627965 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fc5720-3612-4431-9af7-cbcdc928dc0b None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:40.628439 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fc5720-3612-4431-9af7-cbcdc928dc0b None admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:40.629375 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fc5720-3612-4431-9af7-cbcdc928dc0b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:40.629885 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fc5720-3612-4431-9af7-cbcdc928dc0b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:40.631471 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fc5720-3612-4431-9af7-cbcdc928dc0b None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:40.648180 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 268/547] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Aug 10 08:23:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:40.656328 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d66057a-6083-483c-a95b-2247150d6224 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:40.711167 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:40.751829 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eefd2ea-3f40-4e2d-bfec-0b85b04f4f90 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:40.752146 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eefd2ea-3f40-4e2d-bfec-0b85b04f4f90 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:40.752408 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eefd2ea-3f40-4e2d-bfec-0b85b04f4f90 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:40.758386 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 269/548] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:23:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 08:23:40.762261 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2407433-7319-4485-ac68-cf1118badf47 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:40.762546 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2407433-7319-4485-ac68-cf1118badf47 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:40.762822 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2407433-7319-4485-ac68-cf1118badf47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:40.773896 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d66057a-6083-483c-a95b-2247150d6224 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:40.801925 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-a2407433-7319-4485-ac68-cf1118badf47 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:23:40.803135 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-a2407433-7319-4485-ac68-cf1118badf47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:23:40.808009 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10c9804-472e-418a-b573-3be37f0cceb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:40.808647 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10c9804-472e-418a-b573-3be37f0cceb7 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:40.808944 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10c9804-472e-418a-b573-3be37f0cceb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:40.809310 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10c9804-472e-418a-b573-3be37f0cceb7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:40.818144 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10c9804-472e-418a-b573-3be37f0cceb7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:40.819605 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10c9804-472e-418a-b573-3be37f0cceb7 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:40.831326 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-a10c9804-472e-418a-b573-3be37f0cceb7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 10 08:23:40.831326 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:40.831326 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:40.831326 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:40.831326 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:40.831326 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:40.831326 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:40.831326 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:40.831326 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:40.831326 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:40.831326 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:40.831326 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:40.831326 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 08:23:40.831326 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 08:23:40.831326 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 08:23:40.831326 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 08:23:40.833132 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:40.833132 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:40.833132 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 08:23:40.833132 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 08:23:40.833132 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 08:23:40.833132 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 08:23:40.833132 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:40.833132 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:40.833132 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:40.833132 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:40.833132 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:40.833132 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:40.833132 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:40.833132 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:40.833132 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:40.833132 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:40.833132 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:40.833132 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:40.834189 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:40.834189 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:40.834189 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 08:23:40.834189 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 08:23:40.834189 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 08:23:40.834189 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 08:23:40.834189 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 08:23:40.834189 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 08:23:40.834189 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 10 08:23:40.834189 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:23:40.834189 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 280/549] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Aug 10 08:23:40 2022] GET /identity/v3/users/swift => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:40.839183 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10c9804-472e-418a-b573-3be37f0cceb7 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:40.932710 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:40.953034 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10c9804-472e-418a-b573-3be37f0cceb7 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:40.962364 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:41.001566 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 270/550] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:23:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 08:23:41.003802 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1567b667-6a06-4369-96b0-ba419d9004cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:41.004987 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1567b667-6a06-4369-96b0-ba419d9004cd None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:41.005296 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1567b667-6a06-4369-96b0-ba419d9004cd None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:41.005536 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1567b667-6a06-4369-96b0-ba419d9004cd None admin] PATH_INFO: `/v3/users` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:41.006284 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1567b667-6a06-4369-96b0-ba419d9004cd None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:41.006602 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1567b667-6a06-4369-96b0-ba419d9004cd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:41.008078 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1567b667-6a06-4369-96b0-ba419d9004cd None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:41.051701 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 281/551] 10.222.0.57 () {56 vars in 1135 bytes} [Wed Aug 10 08:23:40 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:23:41.056514 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2407433-7319-4485-ac68-cf1118badf47 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:41.075120 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1567b667-6a06-4369-96b0-ba419d9004cd None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:41.213444 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1567b667-6a06-4369-96b0-ba419d9004cd None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:41.258515 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f915749-60ac-43f7-8744-a4eeee12a0a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:41.259127 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f915749-60ac-43f7-8744-a4eeee12a0a3 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:41.259413 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f915749-60ac-43f7-8744-a4eeee12a0a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:41.259684 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f915749-60ac-43f7-8744-a4eeee12a0a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:41.289418 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2407433-7319-4485-ac68-cf1118badf47 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:41.305394 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:41.333721 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f915749-60ac-43f7-8744-a4eeee12a0a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:41.335277 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f915749-60ac-43f7-8744-a4eeee12a0a3 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:41.349998 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cf8d30-8022-4bba-a2ca-a3a16a215ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:41.350918 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cf8d30-8022-4bba-a2ca-a3a16a215ce3 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:41.351397 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cf8d30-8022-4bba-a2ca-a3a16a215ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:41.351865 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cf8d30-8022-4bba-a2ca-a3a16a215ce3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:41.374152 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cf8d30-8022-4bba-a2ca-a3a16a215ce3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 08:23:41.388944 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-49cf8d30-8022-4bba-a2ca-a3a16a215ce3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 08:23:41.388944 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:41.388944 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:41.388944 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:41.388944 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:41.388944 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:41.388944 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:41.388944 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:41.388944 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:41.388944 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:41.388944 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:41.388944 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:41.388944 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 08:23:41.388944 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 08:23:41.388944 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 08:23:41.388944 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 08:23:41.389952 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 08:23:41.389952 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 08:23:41.389952 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 08:23:41.389952 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 08:23:41.389952 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:41.389952 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:41.389952 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:41.389952 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:41.389952 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:41.389952 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:41.389952 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:41.389952 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:41.389952 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:41.389952 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:41.389952 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:41.389952 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:41.390774 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:41.390774 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:41.390774 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:41.390774 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:41.390774 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 08:23:41.390774 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 08:23:41.390774 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 08:23:41.390774 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 08:23:41.390774 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 08:23:41.390774 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 08:23:41.390774 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 08:23:41.390774 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:23:41.394272 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 271/552] 10.222.0.57 () {56 vars in 1102 bytes} [Wed Aug 10 08:23:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:41.406623 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cf8d30-8022-4bba-a2ca-a3a16a215ce3 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:41.443463 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 282/553] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Aug 10 08:23:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 10 08:23:41.451601 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f915749-60ac-43f7-8744-a4eeee12a0a3 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:41.482035 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:41.553329 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:41.567834 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f915749-60ac-43f7-8744-a4eeee12a0a3 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:41.602424 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cf8d30-8022-4bba-a2ca-a3a16a215ce3 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:41.608784 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a928cf-6513-4085-aaa1-92b4fe846cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:41.610079 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a928cf-6513-4085-aaa1-92b4fe846cef None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:41.610718 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a928cf-6513-4085-aaa1-92b4fe846cef None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:41.611094 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a928cf-6513-4085-aaa1-92b4fe846cef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:41.627337 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a928cf-6513-4085-aaa1-92b4fe846cef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:41.629965 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a928cf-6513-4085-aaa1-92b4fe846cef None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:41.662530 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-39a928cf-6513-4085-aaa1-92b4fe846cef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:23:41.662530 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:41.662530 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:41.662530 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:41.662530 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:41.662530 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:41.662530 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:41.662530 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:41.662530 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:41.662530 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:41.662530 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:41.662530 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:41.662530 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 08:23:41.662530 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 08:23:41.662530 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 08:23:41.662530 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 08:23:41.663585 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:41.663585 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:41.663585 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:41.663585 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:41.663585 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:41.663585 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:41.663585 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:41.663585 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:41.663585 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:41.663585 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:41.663585 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:41.663585 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:41.663585 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:41.663585 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:41.663585 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:41.663585 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:41.663585 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 08:23:41.664475 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 08:23:41.664475 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:23:41.664475 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:23:41.664475 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 283/554] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:23:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 08:23:41.670569 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a928cf-6513-4085-aaa1-92b4fe846cef None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:41.679496 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b287683-ae1b-4546-9ecf-b0863015bfab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:41.680587 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b287683-ae1b-4546-9ecf-b0863015bfab None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:41.686669 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b287683-ae1b-4546-9ecf-b0863015bfab None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:41.686972 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b287683-ae1b-4546-9ecf-b0863015bfab None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:41.687665 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b287683-ae1b-4546-9ecf-b0863015bfab None admin] RBAC: Adding query filter params (name=service) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:41.687970 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b287683-ae1b-4546-9ecf-b0863015bfab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:41.689555 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b287683-ae1b-4546-9ecf-b0863015bfab None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:41.699946 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 272/555] 10.222.0.57 () {56 vars in 1103 bytes} [Wed Aug 10 08:23:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:23:41.705730 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b287683-ae1b-4546-9ecf-b0863015bfab None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:41.809182 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a928cf-6513-4085-aaa1-92b4fe846cef None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:41.849723 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:41.859552 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:41.863069 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b287683-ae1b-4546-9ecf-b0863015bfab None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:41.883051 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbcc3cf-7234-4396-9706-45a63c979492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:41.883674 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbcc3cf-7234-4396-9706-45a63c979492 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:41.883954 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbcc3cf-7234-4396-9706-45a63c979492 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:41.884226 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbcc3cf-7234-4396-9706-45a63c979492 None admin] PATH_INFO: `/v3/domains` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:41.884863 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbcc3cf-7234-4396-9706-45a63c979492 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:41.885192 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbcc3cf-7234-4396-9706-45a63c979492 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:41.886567 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbcc3cf-7234-4396-9706-45a63c979492 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:41.911772 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 284/556] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Aug 10 08:23:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:41.915676 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbcc3cf-7234-4396-9706-45a63c979492 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:41.937643 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e686f9-ca20-448b-9878-a446fc629d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:41.940981 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e686f9-ca20-448b-9878-a446fc629d5c None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:41.940981 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e686f9-ca20-448b-9878-a446fc629d5c None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:41.940981 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e686f9-ca20-448b-9878-a446fc629d5c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:41.962321 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e686f9-ca20-448b-9878-a446fc629d5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:41.963785 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e686f9-ca20-448b-9878-a446fc629d5c None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:41.994786 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-34e686f9-ca20-448b-9878-a446fc629d5c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:23:41.994786 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:41.994786 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:41.994786 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:41.994786 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:41.994786 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:41.994786 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:41.994786 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:41.994786 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:41.994786 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:41.994786 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:41.994786 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:41.994786 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 08:23:41.994786 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 08:23:41.994786 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 08:23:41.994786 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 08:23:41.996693 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:41.996693 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:41.996693 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:41.996693 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:41.996693 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:41.996693 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:41.996693 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:41.996693 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:41.996693 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:41.996693 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:41.996693 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:41.996693 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:41.996693 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:41.996693 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:41.996693 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:41.996693 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:41.996693 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 08:23:41.999138 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 08:23:41.999138 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:23:41.999138 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:23:42.000176 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 273/557] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:23:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:42.007539 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e686f9-ca20-448b-9878-a446fc629d5c None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:42.136200 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbcc3cf-7234-4396-9706-45a63c979492 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:42.183360 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c85d8a-94f2-46a1-81bc-fa69e9a887a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:42.183990 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c85d8a-94f2-46a1-81bc-fa69e9a887a8 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:42.184653 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c85d8a-94f2-46a1-81bc-fa69e9a887a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:42.184938 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c85d8a-94f2-46a1-81bc-fa69e9a887a8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:42.193288 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c85d8a-94f2-46a1-81bc-fa69e9a887a8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:42.195012 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c85d8a-94f2-46a1-81bc-fa69e9a887a8 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:42.205991 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e686f9-ca20-448b-9878-a446fc629d5c None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:42.210810 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-47c85d8a-94f2-46a1-81bc-fa69e9a887a8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 08:23:42.210810 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:42.210810 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:42.210810 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:42.210810 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:42.210810 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:42.210810 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:42.210810 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:42.210810 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:42.210810 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:42.210810 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:42.210810 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:42.210810 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 08:23:42.210810 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 08:23:42.210810 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 08:23:42.211699 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 08:23:42.211699 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:42.211699 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:42.211699 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:42.211699 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:42.211699 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:42.211699 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:42.211699 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:42.211699 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:42.211699 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:42.211699 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:42.211699 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:42.211699 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:42.211699 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:42.211699 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:42.211699 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:42.211699 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:42.212575 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 08:23:42.212575 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 08:23:42.212575 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 08:23:42.212575 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 08:23:42.212575 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 08:23:42.212575 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 08:23:42.212575 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 08:23:42.212575 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:23:42.212575 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 285/558] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Aug 10 08:23:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:42.213598 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe6ac50-bbf8-4d80-9bf7-74c2b8a1437e None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:42.216332 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe6ac50-bbf8-4d80-9bf7-74c2b8a1437e None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:42.216332 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe6ac50-bbf8-4d80-9bf7-74c2b8a1437e None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:42.216332 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 286/559] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:23:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:23:42.220035 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe6ac50-bbf8-4d80-9bf7-74c2b8a1437e None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:42.256319 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31d5070-1eb0-43bf-a08e-8ab73ad1051b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:42.256938 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31d5070-1eb0-43bf-a08e-8ab73ad1051b None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:42.257398 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31d5070-1eb0-43bf-a08e-8ab73ad1051b None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:42.257657 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31d5070-1eb0-43bf-a08e-8ab73ad1051b None admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:42.258298 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31d5070-1eb0-43bf-a08e-8ab73ad1051b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:42.258594 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31d5070-1eb0-43bf-a08e-8ab73ad1051b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:42.259963 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31d5070-1eb0-43bf-a08e-8ab73ad1051b None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:42.272013 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:42.273744 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 274/560] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Aug 10 08:23:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:23:42.274402 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb2beb9-0421-4eb3-8701-2a9eaf96d226 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:42.274681 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb2beb9-0421-4eb3-8701-2a9eaf96d226 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:42.274949 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb2beb9-0421-4eb3-8701-2a9eaf96d226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:42.293435 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-feb2beb9-0421-4eb3-8701-2a9eaf96d226 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:23:42.294665 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-feb2beb9-0421-4eb3-8701-2a9eaf96d226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:23:42.297524 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:42.348924 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe6ac50-bbf8-4d80-9bf7-74c2b8a1437e None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:42.376979 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 275/561] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:23:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 08:23:42.376979 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb2beb9-0421-4eb3-8701-2a9eaf96d226 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:42.400085 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0974d930-0eb7-406f-85f9-d3ab0b3a0038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:42.400611 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0974d930-0eb7-406f-85f9-d3ab0b3a0038 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:42.400889 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0974d930-0eb7-406f-85f9-d3ab0b3a0038 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:42.401184 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0974d930-0eb7-406f-85f9-d3ab0b3a0038 None admin] PATH_INFO: `/v3/projects` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:42.404047 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0974d930-0eb7-406f-85f9-d3ab0b3a0038 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:42.404047 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0974d930-0eb7-406f-85f9-d3ab0b3a0038 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:42.404047 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0974d930-0eb7-406f-85f9-d3ab0b3a0038 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:42.415817 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 287/562] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Aug 10 08:23:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:42.417202 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f635a52c-0cc2-43f4-b078-81f252e80771 None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:42.417644 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f635a52c-0cc2-43f4-b078-81f252e80771 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:42.418040 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f635a52c-0cc2-43f4-b078-81f252e80771 None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:42.419555 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 288/563] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:23:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:23:42.424641 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f635a52c-0cc2-43f4-b078-81f252e80771 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:42.507274 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb2beb9-0421-4eb3-8701-2a9eaf96d226 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:42.531362 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:42.538902 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f635a52c-0cc2-43f4-b078-81f252e80771 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:42.543280 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a896fec-c7fc-433e-98b0-be7629e32cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:42.544069 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a896fec-c7fc-433e-98b0-be7629e32cc1 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:42.544424 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a896fec-c7fc-433e-98b0-be7629e32cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:42.544836 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a896fec-c7fc-433e-98b0-be7629e32cc1 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:42.552848 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a896fec-c7fc-433e-98b0-be7629e32cc1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:42.554411 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a896fec-c7fc-433e-98b0-be7629e32cc1 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:42.557121 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:42.567077 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-4a896fec-c7fc-433e-98b0-be7629e32cc1 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 10 08:23:42.567077 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:42.567077 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:42.567077 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:42.567077 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:42.567077 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:42.567077 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:42.567077 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:42.567077 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:42.567077 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:42.567077 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:42.567077 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:42.567077 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 08:23:42.567077 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 08:23:42.567077 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 08:23:42.567077 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 08:23:42.569026 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:42.569026 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:42.569026 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 08:23:42.569026 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 08:23:42.569026 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 08:23:42.569026 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 08:23:42.569026 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:42.569026 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:42.569026 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:42.569026 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:42.569026 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:42.569026 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:42.569026 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:42.569026 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:42.569026 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:42.569026 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:42.569026 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:42.569026 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:42.570023 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:42.570023 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:42.570023 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 08:23:42.570023 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 08:23:42.570023 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 08:23:42.570023 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 08:23:42.570023 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 08:23:42.570023 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 08:23:42.570023 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 10 08:23:42.570023 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:23:42.570023 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 276/564] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Aug 10 08:23:42 2022] GET /identity/v3/users/glance => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:42.570023 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a896fec-c7fc-433e-98b0-be7629e32cc1 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:42.585246 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2875c536-ecec-40e5-ab6b-031c04bb412c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:42.585886 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2875c536-ecec-40e5-ab6b-031c04bb412c None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:42.586327 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2875c536-ecec-40e5-ab6b-031c04bb412c None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:42.586651 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2875c536-ecec-40e5-ab6b-031c04bb412c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:42.597778 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2875c536-ecec-40e5-ab6b-031c04bb412c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 08:23:42.602564 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-2875c536-ecec-40e5-ab6b-031c04bb412c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 08:23:42.602564 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:42.602564 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:42.602564 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:42.602564 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:42.602564 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:42.602564 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:42.602564 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:42.602564 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:42.602564 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:42.602564 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:42.602564 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:42.602564 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 08:23:42.602564 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 08:23:42.602564 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 08:23:42.602564 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 08:23:42.603515 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 08:23:42.603515 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 08:23:42.603515 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 08:23:42.603515 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 08:23:42.603515 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:42.603515 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:42.603515 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:42.603515 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:42.603515 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:42.603515 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:42.603515 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:42.603515 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:42.603515 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:42.603515 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:42.603515 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:42.603515 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:42.604399 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:42.604399 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:42.604399 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:42.604399 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:42.604399 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 08:23:42.604399 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 08:23:42.604399 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 08:23:42.604399 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 08:23:42.604399 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 08:23:42.604399 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 08:23:42.604399 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 08:23:42.604399 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:23:42.604399 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 289/565] 10.222.0.57 () {56 vars in 1102 bytes} [Wed Aug 10 08:23:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:42.605140 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeed0cd-cfa5-49ba-b3e2-386da3458019 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:42.605140 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeed0cd-cfa5-49ba-b3e2-386da3458019 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:42.606059 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeed0cd-cfa5-49ba-b3e2-386da3458019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:42.623826 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-efeed0cd-cfa5-49ba-b3e2-386da3458019 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:23:42.625055 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-efeed0cd-cfa5-49ba-b3e2-386da3458019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:23:42.684284 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a896fec-c7fc-433e-98b0-be7629e32cc1 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:42.708953 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 290/566] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:23:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 08:23:42.708953 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeed0cd-cfa5-49ba-b3e2-386da3458019 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:42.735859 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e83f1e-9a36-4a95-a066-c712e9e935bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:42.736469 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e83f1e-9a36-4a95-a066-c712e9e935bc None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:42.736748 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e83f1e-9a36-4a95-a066-c712e9e935bc None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:42.737058 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e83f1e-9a36-4a95-a066-c712e9e935bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:42.737785 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e83f1e-9a36-4a95-a066-c712e9e935bc None admin] RBAC: Adding query filter params (role.id=73fdd60e2f804ee69b76fa9145a472ff, scope.project.id=b49e381419e940278bcb6111899589b5, user.id=b4b26ba3a9e04c38b52db2a46198a53c) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:42.738092 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e83f1e-9a36-4a95-a066-c712e9e935bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:42.740108 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e83f1e-9a36-4a95-a066-c712e9e935bc None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:42.746148 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 277/567] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Aug 10 08:23:42 2022] GET /identity/v3/role_assignments?user.id=b4b26ba3a9e04c38b52db2a46198a53c&scope.project.id=b49e381419e940278bcb6111899589b5&role.id=73fdd60e2f804ee69b76fa9145a472ff => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:42.746769 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e83f1e-9a36-4a95-a066-c712e9e935bc None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:42.811427 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:42.830157 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeed0cd-cfa5-49ba-b3e2-386da3458019 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:42.842582 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:42.873655 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e88a94-4539-43e8-af48-e6b57b9d7469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:42.874283 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e88a94-4539-43e8-af48-e6b57b9d7469 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:42.875095 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e88a94-4539-43e8-af48-e6b57b9d7469 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:42.875393 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e88a94-4539-43e8-af48-e6b57b9d7469 None admin] PATH_INFO: `/v3/users` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:42.876150 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e88a94-4539-43e8-af48-e6b57b9d7469 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:42.876570 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e88a94-4539-43e8-af48-e6b57b9d7469 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:42.878085 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e88a94-4539-43e8-af48-e6b57b9d7469 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:42.889001 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e83f1e-9a36-4a95-a066-c712e9e935bc None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:42.899542 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 291/568] 10.222.0.57 () {56 vars in 1137 bytes} [Wed Aug 10 08:23:42 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:42.900641 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e88a94-4539-43e8-af48-e6b57b9d7469 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:42.932295 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4146a279-16b5-4232-aae1-951a83c28750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:42.932902 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4146a279-16b5-4232-aae1-951a83c28750 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:42.933203 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4146a279-16b5-4232-aae1-951a83c28750 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:42.933473 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4146a279-16b5-4232-aae1-951a83c28750 None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:42.934189 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4146a279-16b5-4232-aae1-951a83c28750 None admin] RBAC: Adding query filter params (name=service) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:42.934555 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4146a279-16b5-4232-aae1-951a83c28750 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:42.935858 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4146a279-16b5-4232-aae1-951a83c28750 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:42.951253 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 278/569] 10.222.0.57 () {56 vars in 1103 bytes} [Wed Aug 10 08:23:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:42.951253 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c22421-5aca-43a7-a824-f099cd578d6c None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:42.951253 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c22421-5aca-43a7-a824-f099cd578d6c None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:42.951253 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c22421-5aca-43a7-a824-f099cd578d6c None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:42.951253 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 279/570] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:23:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:23:42.951253 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c22421-5aca-43a7-a824-f099cd578d6c None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:43.034584 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e88a94-4539-43e8-af48-e6b57b9d7469 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:43.071246 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c22421-5aca-43a7-a824-f099cd578d6c None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:43.072438 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66881ade-bdee-4b9e-bb24-47130c000aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:43.073225 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66881ade-bdee-4b9e-bb24-47130c000aa1 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:43.073605 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66881ade-bdee-4b9e-bb24-47130c000aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:43.073966 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66881ade-bdee-4b9e-bb24-47130c000aa1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:43.085447 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66881ade-bdee-4b9e-bb24-47130c000aa1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 08:23:43.090193 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-66881ade-bdee-4b9e-bb24-47130c000aa1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 08:23:43.090193 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:43.090193 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:43.090193 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:43.090193 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:43.090193 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:43.090193 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:43.090193 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:43.090193 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:43.090193 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:43.090193 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:43.090193 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:43.090193 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 08:23:43.090193 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 08:23:43.090193 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 08:23:43.090193 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 08:23:43.091345 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 08:23:43.091345 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 08:23:43.091345 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 08:23:43.091345 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 08:23:43.091345 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:43.091345 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:43.091345 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:43.091345 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:43.091345 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:43.091345 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:43.091345 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:43.091345 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:43.091345 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:43.091345 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:43.091345 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:43.091345 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:43.092437 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:43.092437 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:43.092437 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:43.092437 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:43.092437 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 08:23:43.092437 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 08:23:43.092437 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 08:23:43.092437 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 08:23:43.092437 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 08:23:43.092437 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 08:23:43.092437 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 08:23:43.092437 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:23:43.093820 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 292/571] 10.222.0.57 () {56 vars in 1102 bytes} [Wed Aug 10 08:23:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:43.098695 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66881ade-bdee-4b9e-bb24-47130c000aa1 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:43.102698 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:43.104960 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:43.113879 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d967aa92-b7cd-4837-81f1-5bf7887206e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:43.114188 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d967aa92-b7cd-4837-81f1-5bf7887206e1 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:43.114469 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d967aa92-b7cd-4837-81f1-5bf7887206e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:43.114739 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d967aa92-b7cd-4837-81f1-5bf7887206e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:43.126454 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d967aa92-b7cd-4837-81f1-5bf7887206e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:43.127907 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d967aa92-b7cd-4837-81f1-5bf7887206e1 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:43.143009 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-d967aa92-b7cd-4837-81f1-5bf7887206e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:23:43.143009 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:43.143009 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:43.143009 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:43.143009 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:43.143009 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:43.143009 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:43.143009 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:43.143009 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:43.143009 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:43.143009 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:43.143009 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:43.143009 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 08:23:43.143009 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 08:23:43.143009 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 08:23:43.143009 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 08:23:43.150077 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:43.150077 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:43.150077 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:43.150077 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:43.150077 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:43.150077 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:43.150077 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:43.150077 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:43.150077 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:43.150077 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:43.150077 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:43.150077 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:43.150077 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:43.150077 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:43.150077 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:43.150077 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:43.150077 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 08:23:43.151228 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 08:23:43.151228 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:23:43.151228 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:23:43.151228 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 280/572] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:23:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:43.151228 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0043937a-ab2b-4430-8715-6b3d9bdad920 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:43.151228 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0043937a-ab2b-4430-8715-6b3d9bdad920 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:43.151228 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0043937a-ab2b-4430-8715-6b3d9bdad920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:43.167026 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-0043937a-ab2b-4430-8715-6b3d9bdad920 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:23:43.168203 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-0043937a-ab2b-4430-8715-6b3d9bdad920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:23:43.214568 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66881ade-bdee-4b9e-bb24-47130c000aa1 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:43.236457 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 281/573] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:23:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 08:23:43.237337 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0043937a-ab2b-4430-8715-6b3d9bdad920 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:43.260531 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46239f33-965e-4637-a5c7-60b9f8e5aefb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:43.261733 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46239f33-965e-4637-a5c7-60b9f8e5aefb None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:43.262176 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46239f33-965e-4637-a5c7-60b9f8e5aefb None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:43.262997 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46239f33-965e-4637-a5c7-60b9f8e5aefb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:43.275499 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46239f33-965e-4637-a5c7-60b9f8e5aefb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:43.277081 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46239f33-965e-4637-a5c7-60b9f8e5aefb None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:43.293230 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-46239f33-965e-4637-a5c7-60b9f8e5aefb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:23:43.293230 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:43.293230 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:43.293230 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:43.293230 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:43.293230 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:43.293230 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:43.293230 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:43.293230 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:43.293230 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:43.293230 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:43.293230 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:43.293230 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 08:23:43.293230 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 08:23:43.293230 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 08:23:43.293230 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 08:23:43.294229 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:43.294229 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:43.294229 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:43.294229 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:43.294229 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:43.294229 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:43.294229 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:43.294229 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:43.294229 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:43.294229 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:43.294229 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:43.294229 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:43.294229 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:43.294229 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:43.294229 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:43.294229 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:43.294229 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 08:23:43.295127 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 08:23:43.295127 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:23:43.295127 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:23:43.295127 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 293/574] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:23:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:43.296024 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46239f33-965e-4637-a5c7-60b9f8e5aefb None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:43.354238 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:43.359487 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0043937a-ab2b-4430-8715-6b3d9bdad920 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:43.369883 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:43.394981 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d381eda7-cdd8-4b3e-bbe3-f86fb4468c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:43.395783 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d381eda7-cdd8-4b3e-bbe3-f86fb4468c65 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:43.396068 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d381eda7-cdd8-4b3e-bbe3-f86fb4468c65 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:43.396332 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d381eda7-cdd8-4b3e-bbe3-f86fb4468c65 None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:43.397029 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d381eda7-cdd8-4b3e-bbe3-f86fb4468c65 None admin] RBAC: Adding query filter params (name=service) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:43.397332 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d381eda7-cdd8-4b3e-bbe3-f86fb4468c65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:43.399025 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d381eda7-cdd8-4b3e-bbe3-f86fb4468c65 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:43.408127 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 282/575] 10.222.0.57 () {56 vars in 1103 bytes} [Wed Aug 10 08:23:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:43.409260 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d381eda7-cdd8-4b3e-bbe3-f86fb4468c65 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:43.413767 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46239f33-965e-4637-a5c7-60b9f8e5aefb None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:43.448848 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd659fc-3eb4-4aa3-a1be-2cdc8083944c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:43.449484 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd659fc-3eb4-4aa3-a1be-2cdc8083944c None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:43.449759 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd659fc-3eb4-4aa3-a1be-2cdc8083944c None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:43.450026 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd659fc-3eb4-4aa3-a1be-2cdc8083944c None admin] PATH_INFO: `/v3/domains` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:43.450665 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd659fc-3eb4-4aa3-a1be-2cdc8083944c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:43.450964 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd659fc-3eb4-4aa3-a1be-2cdc8083944c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:43.452321 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd659fc-3eb4-4aa3-a1be-2cdc8083944c None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:43.469341 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 294/576] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Aug 10 08:23:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:43.469341 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd659fc-3eb4-4aa3-a1be-2cdc8083944c None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:43.527652 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d381eda7-cdd8-4b3e-bbe3-f86fb4468c65 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:43.565393 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fbcdcf-8caa-47d2-8343-71dece5ca474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:43.566027 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fbcdcf-8caa-47d2-8343-71dece5ca474 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:43.566469 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fbcdcf-8caa-47d2-8343-71dece5ca474 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:43.566660 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fbcdcf-8caa-47d2-8343-71dece5ca474 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:43.571561 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd659fc-3eb4-4aa3-a1be-2cdc8083944c None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:43.577223 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fbcdcf-8caa-47d2-8343-71dece5ca474 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 08:23:43.582947 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-98fbcdcf-8caa-47d2-8343-71dece5ca474 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 08:23:43.582947 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:43.582947 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:43.582947 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:43.582947 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:43.582947 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:43.582947 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:43.582947 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:43.582947 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:43.582947 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:43.582947 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:43.582947 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:43.582947 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 08:23:43.582947 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 08:23:43.582947 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 08:23:43.582947 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 08:23:43.583943 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 08:23:43.583943 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 08:23:43.583943 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 08:23:43.583943 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 08:23:43.583943 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:43.583943 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:43.583943 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:43.583943 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:43.583943 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:43.583943 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:43.583943 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:43.583943 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:43.583943 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:43.583943 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:43.583943 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:43.583943 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:43.584800 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:43.584800 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:43.584800 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:43.584800 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:43.584800 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 08:23:43.584800 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 08:23:43.584800 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 08:23:43.584800 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 08:23:43.584800 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 08:23:43.584800 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 08:23:43.584800 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 08:23:43.584800 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:23:43.584800 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 283/577] 10.222.0.57 () {56 vars in 1102 bytes} [Wed Aug 10 08:23:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:43.584800 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fbcdcf-8caa-47d2-8343-71dece5ca474 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:43.591944 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:43.611447 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90634775-2be7-43ee-bd7c-9ce32429ce47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:43.612047 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90634775-2be7-43ee-bd7c-9ce32429ce47 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:43.612327 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90634775-2be7-43ee-bd7c-9ce32429ce47 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:43.612596 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90634775-2be7-43ee-bd7c-9ce32429ce47 None admin] PATH_INFO: `/v3/domains` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:43.613290 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90634775-2be7-43ee-bd7c-9ce32429ce47 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:43.613640 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90634775-2be7-43ee-bd7c-9ce32429ce47 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:43.615201 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90634775-2be7-43ee-bd7c-9ce32429ce47 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:43.623090 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:43.631885 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 295/578] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Aug 10 08:23:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:43.631885 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90634775-2be7-43ee-bd7c-9ce32429ce47 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:43.705224 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fbcdcf-8caa-47d2-8343-71dece5ca474 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:43.733675 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90634775-2be7-43ee-bd7c-9ce32429ce47 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:43.744530 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8048735-2fdc-42ab-a3d8-a3ce691eebaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:43.745448 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8048735-2fdc-42ab-a3d8-a3ce691eebaa None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:43.745729 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8048735-2fdc-42ab-a3d8-a3ce691eebaa None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:43.745998 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8048735-2fdc-42ab-a3d8-a3ce691eebaa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:43.757886 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8048735-2fdc-42ab-a3d8-a3ce691eebaa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:43.759318 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8048735-2fdc-42ab-a3d8-a3ce691eebaa None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:43.770989 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0a2dbb-5d98-47f5-84e9-b18bdd4e2e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:43.771609 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0a2dbb-5d98-47f5-84e9-b18bdd4e2e19 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:43.771877 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0a2dbb-5d98-47f5-84e9-b18bdd4e2e19 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:43.772141 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0a2dbb-5d98-47f5-84e9-b18bdd4e2e19 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:43.774816 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-e8048735-2fdc-42ab-a3d8-a3ce691eebaa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:23:43.774816 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:43.774816 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:43.774816 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:43.774816 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:43.774816 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:43.774816 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:43.774816 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:43.774816 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:43.774816 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:43.774816 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:43.774816 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:43.774816 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 08:23:43.774816 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 08:23:43.774816 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 08:23:43.774816 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 08:23:43.775773 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:43.775773 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:43.775773 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:43.775773 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:43.775773 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:43.775773 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:43.775773 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:43.775773 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:43.775773 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:43.775773 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:43.775773 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:43.775773 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:43.775773 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:43.775773 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:43.775773 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:43.775773 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:43.775773 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 08:23:43.776699 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 08:23:43.776699 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:23:43.776699 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:23:43.776699 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 284/579] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:23:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:43.778663 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0a2dbb-5d98-47f5-84e9-b18bdd4e2e19 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:43.780092 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0a2dbb-5d98-47f5-84e9-b18bdd4e2e19 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:43.781270 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8048735-2fdc-42ab-a3d8-a3ce691eebaa None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:43.792731 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-9a0a2dbb-5d98-47f5-84e9-b18bdd4e2e19 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 08:23:43.792731 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:43.792731 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:43.792731 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:43.792731 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:43.792731 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:43.792731 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:43.792731 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:43.792731 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:43.792731 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:43.792731 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:43.792731 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:43.792731 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 08:23:43.792731 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 08:23:43.792731 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 08:23:43.793805 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 08:23:43.793805 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:43.793805 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:43.793805 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:43.793805 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:43.793805 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:43.793805 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:43.793805 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:43.793805 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:43.793805 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:43.793805 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:43.793805 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:43.793805 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:43.793805 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:43.793805 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:43.793805 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:43.793805 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:43.794745 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 08:23:43.794745 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 08:23:43.794745 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 08:23:43.794745 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 08:23:43.794745 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 08:23:43.794745 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 08:23:43.794745 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 08:23:43.794745 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:23:43.794745 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 296/580] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Aug 10 08:23:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:43.795828 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0a2dbb-5d98-47f5-84e9-b18bdd4e2e19 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:43.836213 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:43.876628 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:43.895160 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8048735-2fdc-42ab-a3d8-a3ce691eebaa None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:43.904769 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0a2dbb-5d98-47f5-84e9-b18bdd4e2e19 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:43.931864 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fcd43d-9b33-44ca-adbd-fa1db25ce0dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:43.932685 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fcd43d-9b33-44ca-adbd-fa1db25ce0dc None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:43.933267 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fcd43d-9b33-44ca-adbd-fa1db25ce0dc None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:43.933544 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fcd43d-9b33-44ca-adbd-fa1db25ce0dc None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:43.934210 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fcd43d-9b33-44ca-adbd-fa1db25ce0dc None admin] RBAC: Adding query filter params (name=service) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:43.934515 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fcd43d-9b33-44ca-adbd-fa1db25ce0dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:43.935922 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fcd43d-9b33-44ca-adbd-fa1db25ce0dc None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:43.942229 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6ca3fd-e7c7-4189-80e3-ba037b47f7a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:43.942880 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6ca3fd-e7c7-4189-80e3-ba037b47f7a4 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:43.943158 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6ca3fd-e7c7-4189-80e3-ba037b47f7a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:43.943420 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6ca3fd-e7c7-4189-80e3-ba037b47f7a4 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:43.949674 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 285/581] 10.222.0.57 () {56 vars in 1103 bytes} [Wed Aug 10 08:23:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:43.949674 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fcd43d-9b33-44ca-adbd-fa1db25ce0dc None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:43.956749 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6ca3fd-e7c7-4189-80e3-ba037b47f7a4 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:43.958815 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6ca3fd-e7c7-4189-80e3-ba037b47f7a4 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:43.969586 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-9b6ca3fd-e7c7-4189-80e3-ba037b47f7a4 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 10 08:23:43.969586 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:43.969586 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:43.969586 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:43.969586 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:43.969586 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:43.969586 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:43.969586 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:43.969586 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:43.969586 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:43.969586 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:43.969586 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:43.969586 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 08:23:43.969586 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 08:23:43.969586 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 08:23:43.969586 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 08:23:43.970737 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:43.970737 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:43.970737 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 08:23:43.970737 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 08:23:43.970737 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 08:23:43.970737 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 08:23:43.970737 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:43.970737 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:43.970737 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:43.970737 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:43.970737 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:43.970737 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:43.970737 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:43.970737 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:43.970737 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:43.970737 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:43.970737 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:43.970737 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:43.971703 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:43.971703 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:43.971703 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 08:23:43.971703 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 08:23:43.971703 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 08:23:43.971703 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 08:23:43.971703 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 08:23:43.971703 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 08:23:43.971703 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 10 08:23:43.971703 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:23:43.971703 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 297/582] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Aug 10 08:23:43 2022] GET /identity/v3/users/cinder => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:43.971703 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6ca3fd-e7c7-4189-80e3-ba037b47f7a4 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:44.058192 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fcd43d-9b33-44ca-adbd-fa1db25ce0dc None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:44.070757 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:44.076379 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6ca3fd-e7c7-4189-80e3-ba037b47f7a4 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:44.093870 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143fd1cd-112a-4396-9908-3dddcc3c96c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:44.094474 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143fd1cd-112a-4396-9908-3dddcc3c96c0 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:44.094753 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143fd1cd-112a-4396-9908-3dddcc3c96c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:44.095022 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143fd1cd-112a-4396-9908-3dddcc3c96c0 None admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:44.095687 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143fd1cd-112a-4396-9908-3dddcc3c96c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:44.095987 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143fd1cd-112a-4396-9908-3dddcc3c96c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:44.097376 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143fd1cd-112a-4396-9908-3dddcc3c96c0 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:44.114704 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 286/583] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Aug 10 08:23:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:44.114704 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143fd1cd-112a-4396-9908-3dddcc3c96c0 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:44.115948 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:44.121643 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b74c0c-0cac-4a58-a817-1c74c53d48e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:44.122380 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b74c0c-0cac-4a58-a817-1c74c53d48e6 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:44.123001 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b74c0c-0cac-4a58-a817-1c74c53d48e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:44.123158 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b74c0c-0cac-4a58-a817-1c74c53d48e6 None admin] PATH_INFO: `/v3/projects` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:44.123924 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b74c0c-0cac-4a58-a817-1c74c53d48e6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:44.124330 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b74c0c-0cac-4a58-a817-1c74c53d48e6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:44.125841 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b74c0c-0cac-4a58-a817-1c74c53d48e6 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:44.143950 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 298/584] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Aug 10 08:23:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:44.143950 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b74c0c-0cac-4a58-a817-1c74c53d48e6 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:44.224759 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143fd1cd-112a-4396-9908-3dddcc3c96c0 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:44.254934 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b74c0c-0cac-4a58-a817-1c74c53d48e6 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:44.259486 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06984d74-f65d-4283-88ea-6742c944c8b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:44.260116 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06984d74-f65d-4283-88ea-6742c944c8b9 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:44.260395 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06984d74-f65d-4283-88ea-6742c944c8b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:44.260667 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06984d74-f65d-4283-88ea-6742c944c8b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:44.271863 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06984d74-f65d-4283-88ea-6742c944c8b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:44.273439 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06984d74-f65d-4283-88ea-6742c944c8b9 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:44.289751 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-06984d74-f65d-4283-88ea-6742c944c8b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:23:44.289751 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:44.289751 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:44.289751 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:44.289751 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:44.289751 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:44.289751 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:44.289751 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:44.289751 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:44.289751 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:44.289751 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:44.289751 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:44.289751 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 08:23:44.289751 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 08:23:44.289751 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 08:23:44.289751 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 08:23:44.290805 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:44.290805 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:44.290805 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:44.290805 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:44.290805 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:44.290805 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:44.290805 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:44.290805 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:44.290805 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:44.290805 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:44.290805 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:44.290805 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:44.290805 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:44.290805 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:44.290805 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:44.290805 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:44.290805 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 08:23:44.291715 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 08:23:44.291715 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:23:44.291715 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:23:44.291715 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 287/585] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:23:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:44.291715 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06984d74-f65d-4283-88ea-6742c944c8b9 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:44.298887 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de78a370-daad-4229-9145-a9c603244517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:44.299511 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de78a370-daad-4229-9145-a9c603244517 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:44.299793 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de78a370-daad-4229-9145-a9c603244517 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:44.300064 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de78a370-daad-4229-9145-a9c603244517 None admin] PATH_INFO: `/v3/users` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:44.301007 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de78a370-daad-4229-9145-a9c603244517 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:44.301381 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de78a370-daad-4229-9145-a9c603244517 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:44.302869 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de78a370-daad-4229-9145-a9c603244517 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:44.320426 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 299/586] 10.222.0.57 () {56 vars in 1137 bytes} [Wed Aug 10 08:23:44 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:44.321177 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de78a370-daad-4229-9145-a9c603244517 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:44.342230 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:44.357745 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:44.412523 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06984d74-f65d-4283-88ea-6742c944c8b9 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:44.445331 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de78a370-daad-4229-9145-a9c603244517 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:44.450493 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d7087f-802f-4a07-bb5a-b09b837ebe40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:44.451115 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d7087f-802f-4a07-bb5a-b09b837ebe40 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:44.451391 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d7087f-802f-4a07-bb5a-b09b837ebe40 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:44.451658 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d7087f-802f-4a07-bb5a-b09b837ebe40 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:44.459864 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d7087f-802f-4a07-bb5a-b09b837ebe40 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:44.461323 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d7087f-802f-4a07-bb5a-b09b837ebe40 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:44.472638 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-88d7087f-802f-4a07-bb5a-b09b837ebe40 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 08:23:44.472638 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:44.472638 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:44.472638 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:44.472638 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:44.472638 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:44.472638 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:44.472638 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:44.472638 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:44.472638 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:44.472638 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:44.472638 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:44.472638 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 08:23:44.472638 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 08:23:44.472638 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 08:23:44.472638 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 08:23:44.473648 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:44.473648 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:44.473648 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 08:23:44.473648 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 08:23:44.473648 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 08:23:44.473648 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 08:23:44.473648 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:44.473648 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:44.473648 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:44.473648 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:44.473648 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:44.473648 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:44.473648 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:44.473648 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:44.473648 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:44.473648 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:44.473648 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:44.473648 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:44.474603 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:44.474603 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:44.474603 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 08:23:44.474603 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 08:23:44.474603 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 08:23:44.474603 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 08:23:44.474603 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 08:23:44.474603 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 08:23:44.474603 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 08:23:44.474603 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:23:44.474603 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 288/587] 10.222.0.57 () {56 vars in 1093 bytes} [Wed Aug 10 08:23:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:44.475195 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d7087f-802f-4a07-bb5a-b09b837ebe40 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:44.486899 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bad3d4-0128-4b09-aaf1-d8b290907765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:44.487554 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bad3d4-0128-4b09-aaf1-d8b290907765 None admin] REQUEST_METHOD: `PUT` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:44.487829 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bad3d4-0128-4b09-aaf1-d8b290907765 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:44.488174 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bad3d4-0128-4b09-aaf1-d8b290907765 None admin] PATH_INFO: `/v3/projects/b49e381419e940278bcb6111899589b5/users/3263af99fc2f48e79fd1e5b8a1e555a4/roles/73fdd60e2f804ee69b76fa9145a472ff` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:44.503101 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bad3d4-0128-4b09-aaf1-d8b290907765 None admin] RBAC: Authorizing `identity:create_grant(project_id=b49e381419e940278bcb6111899589b5, user_id=3263af99fc2f48e79fd1e5b8a1e555a4, role_id=73fdd60e2f804ee69b76fa9145a472ff)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:44.505506 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bad3d4-0128-4b09-aaf1-d8b290907765 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:44.578321 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 300/588] 10.222.0.57 () {58 vars in 1439 bytes} [Wed Aug 10 08:23:44 2022] PUT /identity/v3/projects/b49e381419e940278bcb6111899589b5/users/3263af99fc2f48e79fd1e5b8a1e555a4/roles/73fdd60e2f804ee69b76fa9145a472ff => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:23:44.579821 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bad3d4-0128-4b09-aaf1-d8b290907765 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:44.583452 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d7087f-802f-4a07-bb5a-b09b837ebe40 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:44.605706 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:44.627127 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa353c66-10e1-4bff-b249-a0d11aac43f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:44.627757 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa353c66-10e1-4bff-b249-a0d11aac43f7 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:44.628035 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa353c66-10e1-4bff-b249-a0d11aac43f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:44.628307 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa353c66-10e1-4bff-b249-a0d11aac43f7 None admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:44.630854 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa353c66-10e1-4bff-b249-a0d11aac43f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:44.631168 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa353c66-10e1-4bff-b249-a0d11aac43f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:44.632858 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa353c66-10e1-4bff-b249-a0d11aac43f7 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:44.655115 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 289/589] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Aug 10 08:23:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:44.655115 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa353c66-10e1-4bff-b249-a0d11aac43f7 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:44.685018 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:44.733303 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bad3d4-0128-4b09-aaf1-d8b290907765 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:44.776813 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5856ce-734e-44f6-9eeb-33866fea878f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:44.777645 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5856ce-734e-44f6-9eeb-33866fea878f None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:44.778035 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5856ce-734e-44f6-9eeb-33866fea878f None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:44.778445 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5856ce-734e-44f6-9eeb-33866fea878f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:44.793752 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5856ce-734e-44f6-9eeb-33866fea878f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:44.794278 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa353c66-10e1-4bff-b249-a0d11aac43f7 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:44.794807 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5856ce-734e-44f6-9eeb-33866fea878f None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:44.814787 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-6c5856ce-734e-44f6-9eeb-33866fea878f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:23:44.814787 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:44.814787 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:44.814787 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:44.814787 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:44.814787 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:44.814787 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:44.814787 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:44.814787 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:44.814787 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:44.814787 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:44.814787 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:44.814787 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 08:23:44.814787 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 08:23:44.814787 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 08:23:44.814787 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 08:23:44.816086 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:44.816086 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:44.816086 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:44.816086 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:44.816086 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:44.816086 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:44.816086 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:44.816086 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:44.816086 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:44.816086 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:44.816086 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:44.816086 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:44.816086 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:44.816086 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:44.816086 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:44.816086 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:44.816086 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 08:23:44.817989 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 08:23:44.817989 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:23:44.817989 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:23:44.819533 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 301/590] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:23:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:44.820426 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5856ce-734e-44f6-9eeb-33866fea878f None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:44.842132 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48523d4a-863a-4321-a40b-072c4b4604e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:44.843016 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48523d4a-863a-4321-a40b-072c4b4604e2 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:44.843448 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48523d4a-863a-4321-a40b-072c4b4604e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:44.843880 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48523d4a-863a-4321-a40b-072c4b4604e2 None admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:44.844760 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48523d4a-863a-4321-a40b-072c4b4604e2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:44.845306 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48523d4a-863a-4321-a40b-072c4b4604e2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:44.846878 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48523d4a-863a-4321-a40b-072c4b4604e2 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:44.867284 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 290/591] 10.222.0.57 () {56 vars in 1133 bytes} [Wed Aug 10 08:23:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:44.868384 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48523d4a-863a-4321-a40b-072c4b4604e2 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:44.910622 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:44.965530 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5856ce-734e-44f6-9eeb-33866fea878f None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:44.985947 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48523d4a-863a-4321-a40b-072c4b4604e2 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:44.989836 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:45.014400 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad50ad63-c943-448f-bd35-2eee5d5fae87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:45.015189 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad50ad63-c943-448f-bd35-2eee5d5fae87 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:45.015794 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad50ad63-c943-448f-bd35-2eee5d5fae87 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:45.016167 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad50ad63-c943-448f-bd35-2eee5d5fae87 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:45.026359 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9377bd-5bfb-4905-bfeb-ba72c769f9a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:45.026572 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad50ad63-c943-448f-bd35-2eee5d5fae87 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:45.027308 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9377bd-5bfb-4905-bfeb-ba72c769f9a2 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:45.027736 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9377bd-5bfb-4905-bfeb-ba72c769f9a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:45.028154 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9377bd-5bfb-4905-bfeb-ba72c769f9a2 None admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:45.028440 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad50ad63-c943-448f-bd35-2eee5d5fae87 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:45.029312 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9377bd-5bfb-4905-bfeb-ba72c769f9a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:45.029819 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9377bd-5bfb-4905-bfeb-ba72c769f9a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:45.031452 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9377bd-5bfb-4905-bfeb-ba72c769f9a2 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:45.044111 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-ad50ad63-c943-448f-bd35-2eee5d5fae87 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 10 08:23:45.044111 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:45.044111 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:45.044111 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:45.044111 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:45.044111 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:45.044111 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:45.044111 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:45.044111 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:45.044111 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:45.044111 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:45.044111 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:45.044111 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 08:23:45.044111 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 08:23:45.044111 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 08:23:45.044111 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 08:23:45.045370 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:45.045370 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:45.045370 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 08:23:45.045370 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 08:23:45.045370 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 08:23:45.045370 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 08:23:45.045370 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:45.045370 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:45.045370 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:45.045370 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:45.045370 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:45.045370 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:45.045370 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:45.045370 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:45.045370 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:45.045370 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:45.045370 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:45.045370 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:45.046615 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:45.046615 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:45.046615 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 08:23:45.046615 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 08:23:45.046615 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 08:23:45.046615 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 08:23:45.046615 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 08:23:45.046615 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 08:23:45.046615 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 10 08:23:45.046615 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:23:45.053265 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 302/593] 10.222.0.57 () {56 vars in 1102 bytes} [Wed Aug 10 08:23:44 2022] GET /identity/v3/users/neutron => generated 85 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:45.053475 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 291/593] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Aug 10 08:23:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:45.054370 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad50ad63-c943-448f-bd35-2eee5d5fae87 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:45.054690 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9377bd-5bfb-4905-bfeb-ba72c769f9a2 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:45.157455 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:45.161868 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9377bd-5bfb-4905-bfeb-ba72c769f9a2 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:45.200249 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6776f196-91be-4ef7-ac1f-415c244bfe9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:45.201180 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6776f196-91be-4ef7-ac1f-415c244bfe9a None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:45.201712 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6776f196-91be-4ef7-ac1f-415c244bfe9a None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:45.202168 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6776f196-91be-4ef7-ac1f-415c244bfe9a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:45.207540 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad50ad63-c943-448f-bd35-2eee5d5fae87 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:45.210928 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6776f196-91be-4ef7-ac1f-415c244bfe9a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:45.213835 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6776f196-91be-4ef7-ac1f-415c244bfe9a None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:45.229197 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-6776f196-91be-4ef7-ac1f-415c244bfe9a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 08:23:45.229197 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:45.229197 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:45.229197 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:45.229197 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:45.229197 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:45.229197 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:45.229197 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:45.229197 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:45.229197 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:45.229197 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:45.229197 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:45.229197 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 08:23:45.229197 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 08:23:45.229197 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 08:23:45.230163 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 08:23:45.230163 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:45.230163 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:45.230163 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:45.230163 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:45.230163 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:45.230163 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:45.230163 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:45.230163 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:45.230163 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:45.230163 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:45.230163 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:45.230163 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:45.230163 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:45.230163 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:45.230163 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:45.230163 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:45.231062 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 08:23:45.231062 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 08:23:45.231062 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 08:23:45.231062 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 08:23:45.231062 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 08:23:45.231062 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 08:23:45.231062 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 08:23:45.231062 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:23:45.238685 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 292/594] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Aug 10 08:23:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:45.239772 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6776f196-91be-4ef7-ac1f-415c244bfe9a None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:45.247802 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3c96ee-a4fd-4141-b485-3168223b3508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:45.248473 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3c96ee-a4fd-4141-b485-3168223b3508 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:45.248753 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3c96ee-a4fd-4141-b485-3168223b3508 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:45.249085 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3c96ee-a4fd-4141-b485-3168223b3508 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:45.263166 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3c96ee-a4fd-4141-b485-3168223b3508 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:45.264638 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3c96ee-a4fd-4141-b485-3168223b3508 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:45.283552 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-ba3c96ee-a4fd-4141-b485-3168223b3508 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:23:45.283552 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:45.283552 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:45.283552 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:45.283552 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:45.283552 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:45.283552 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:45.283552 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:45.283552 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:45.283552 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:45.283552 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:45.283552 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:45.283552 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 08:23:45.283552 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 08:23:45.283552 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 08:23:45.283552 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 08:23:45.284560 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:45.284560 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:45.284560 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:45.284560 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:45.284560 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:45.284560 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:45.284560 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:45.284560 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:45.284560 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:45.284560 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:45.284560 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:45.284560 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:45.284560 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:45.284560 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:45.284560 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:45.284560 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:45.284560 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 08:23:45.286055 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 08:23:45.286055 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:23:45.286055 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:23:45.286055 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 303/595] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:23:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:45.286708 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3c96ee-a4fd-4141-b485-3168223b3508 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:45.320824 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:45.354364 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6776f196-91be-4ef7-ac1f-415c244bfe9a None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:45.394424 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2176c411-2b6a-4df0-9799-18dc94c07107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:45.395220 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2176c411-2b6a-4df0-9799-18dc94c07107 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:45.395500 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2176c411-2b6a-4df0-9799-18dc94c07107 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:45.395760 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2176c411-2b6a-4df0-9799-18dc94c07107 None admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:45.396057 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3c96ee-a4fd-4141-b485-3168223b3508 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:45.396492 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2176c411-2b6a-4df0-9799-18dc94c07107 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:45.396793 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2176c411-2b6a-4df0-9799-18dc94c07107 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:45.401011 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2176c411-2b6a-4df0-9799-18dc94c07107 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:45.424643 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 293/596] 10.222.0.57 () {56 vars in 1139 bytes} [Wed Aug 10 08:23:45 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:45.424643 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8f64c3-cac7-4f9d-ae5f-ff1f5aa560e5 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:45.424643 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8f64c3-cac7-4f9d-ae5f-ff1f5aa560e5 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:45.424643 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8f64c3-cac7-4f9d-ae5f-ff1f5aa560e5 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:45.424643 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 294/597] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:23:45 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:23:45.424643 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8f64c3-cac7-4f9d-ae5f-ff1f5aa560e5 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:45.439632 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5747e476-8e88-400a-bf9e-f86d3bc65550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:45.440313 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5747e476-8e88-400a-bf9e-f86d3bc65550 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:45.440681 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5747e476-8e88-400a-bf9e-f86d3bc65550 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:45.441051 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5747e476-8e88-400a-bf9e-f86d3bc65550 None admin] PATH_INFO: `/v3/projects` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:45.441857 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:45.441968 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5747e476-8e88-400a-bf9e-f86d3bc65550 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:45.442341 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5747e476-8e88-400a-bf9e-f86d3bc65550 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:45.443841 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5747e476-8e88-400a-bf9e-f86d3bc65550 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:45.456772 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 304/598] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Aug 10 08:23:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:45.457818 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5747e476-8e88-400a-bf9e-f86d3bc65550 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:45.539202 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8f64c3-cac7-4f9d-ae5f-ff1f5aa560e5 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:45.554150 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:45.565915 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5747e476-8e88-400a-bf9e-f86d3bc65550 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:45.574405 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf7755c-4dbc-4f72-81eb-1bf5b8a948a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:45.575003 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf7755c-4dbc-4f72-81eb-1bf5b8a948a0 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:45.575539 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf7755c-4dbc-4f72-81eb-1bf5b8a948a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:45.575539 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf7755c-4dbc-4f72-81eb-1bf5b8a948a0 None admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:45.576158 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf7755c-4dbc-4f72-81eb-1bf5b8a948a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:45.576454 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf7755c-4dbc-4f72-81eb-1bf5b8a948a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:45.577834 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf7755c-4dbc-4f72-81eb-1bf5b8a948a0 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:45.589987 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 295/599] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Aug 10 08:23:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:45.589987 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d050e56-b76e-44f1-8962-7e55b2eb2e51 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:45.589987 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d050e56-b76e-44f1-8962-7e55b2eb2e51 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:45.589987 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d050e56-b76e-44f1-8962-7e55b2eb2e51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:45.607395 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45034fc-28de-4277-a8ad-7fe5ef0f5657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:45.608023 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45034fc-28de-4277-a8ad-7fe5ef0f5657 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:45.608301 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45034fc-28de-4277-a8ad-7fe5ef0f5657 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:45.608568 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45034fc-28de-4277-a8ad-7fe5ef0f5657 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:45.610682 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-4d050e56-b76e-44f1-8962-7e55b2eb2e51 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:23:45.611858 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-4d050e56-b76e-44f1-8962-7e55b2eb2e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:23:45.619098 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45034fc-28de-4277-a8ad-7fe5ef0f5657 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:45.621039 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45034fc-28de-4277-a8ad-7fe5ef0f5657 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:45.635691 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-d45034fc-28de-4277-a8ad-7fe5ef0f5657 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:23:45.635691 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:45.635691 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:45.635691 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:45.635691 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:45.635691 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:45.635691 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:45.635691 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:45.635691 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:45.635691 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:45.635691 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:45.635691 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:45.635691 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 08:23:45.635691 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 08:23:45.635691 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 08:23:45.635691 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 08:23:45.636653 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:45.636653 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:45.636653 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:45.636653 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:45.636653 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:45.636653 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:45.636653 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:45.636653 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:45.636653 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:45.636653 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:45.636653 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:45.636653 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:45.636653 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:45.636653 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:45.636653 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:45.636653 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:45.636653 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 08:23:45.637599 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 08:23:45.637599 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:23:45.637599 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:23:45.637599 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 305/600] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:23:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:45.637599 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45034fc-28de-4277-a8ad-7fe5ef0f5657 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:45.688641 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 296/601] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:23:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 08:23:45.689907 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d050e56-b76e-44f1-8962-7e55b2eb2e51 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:45.692422 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:45.764343 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45034fc-28de-4277-a8ad-7fe5ef0f5657 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:45.807768 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d050e56-b76e-44f1-8962-7e55b2eb2e51 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:45.808091 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97309e68-f2fb-4dc7-8fa4-6e8c22e5c2da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:45.808962 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97309e68-f2fb-4dc7-8fa4-6e8c22e5c2da None admin] REQUEST_METHOD: `PUT` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:45.809652 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97309e68-f2fb-4dc7-8fa4-6e8c22e5c2da None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:45.810962 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97309e68-f2fb-4dc7-8fa4-6e8c22e5c2da None admin] PATH_INFO: `/v3/projects/b49e381419e940278bcb6111899589b5/users/f1ba74e28ced4cbba857f54222dfbe41/roles/73fdd60e2f804ee69b76fa9145a472ff` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:45.822865 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:45.826791 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97309e68-f2fb-4dc7-8fa4-6e8c22e5c2da None admin] RBAC: Authorizing `identity:create_grant(project_id=b49e381419e940278bcb6111899589b5, user_id=f1ba74e28ced4cbba857f54222dfbe41, role_id=73fdd60e2f804ee69b76fa9145a472ff)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:45.828305 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97309e68-f2fb-4dc7-8fa4-6e8c22e5c2da None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:45.843204 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609b4f5b-4030-4623-8f1f-2f578e4d07ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:45.849423 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609b4f5b-4030-4623-8f1f-2f578e4d07ab None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:45.849423 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609b4f5b-4030-4623-8f1f-2f578e4d07ab None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:45.849423 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609b4f5b-4030-4623-8f1f-2f578e4d07ab None admin] PATH_INFO: `/v3/projects/service` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:45.851602 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609b4f5b-4030-4623-8f1f-2f578e4d07ab None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:45.853055 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609b4f5b-4030-4623-8f1f-2f578e4d07ab None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:45.863838 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-609b4f5b-4030-4623-8f1f-2f578e4d07ab None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 08:23:45.863838 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:45.863838 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:45.863838 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:45.863838 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:45.863838 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:45.863838 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:45.863838 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:45.863838 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:45.863838 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:45.863838 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:45.863838 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:45.863838 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 08:23:45.863838 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 08:23:45.863838 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 08:23:45.864697 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 08:23:45.864697 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:45.864697 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:45.864697 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:45.864697 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:45.864697 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:45.864697 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:45.864697 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:45.864697 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:45.864697 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:45.864697 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:45.864697 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:45.864697 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:45.864697 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:45.864697 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:45.864697 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:45.864697 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:45.865637 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 08:23:45.865637 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 08:23:45.865637 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 08:23:45.865637 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 08:23:45.865637 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 08:23:45.865637 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 08:23:45.865637 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 08:23:45.865637 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:23:45.865637 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 297/602] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Aug 10 08:23:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:45.866534 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609b4f5b-4030-4623-8f1f-2f578e4d07ab None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:45.906285 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 306/603] 10.222.0.57 () {58 vars in 1439 bytes} [Wed Aug 10 08:23:45 2022] PUT /identity/v3/projects/b49e381419e940278bcb6111899589b5/users/f1ba74e28ced4cbba857f54222dfbe41/roles/73fdd60e2f804ee69b76fa9145a472ff => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:23:45.907455 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97309e68-f2fb-4dc7-8fa4-6e8c22e5c2da None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:45.945396 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:45.994002 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609b4f5b-4030-4623-8f1f-2f578e4d07ab None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:46.034342 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c45350-1ed5-410f-b1af-5eb422b95aa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:46.036014 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c45350-1ed5-410f-b1af-5eb422b95aa0 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:46.036514 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c45350-1ed5-410f-b1af-5eb422b95aa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:46.036999 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c45350-1ed5-410f-b1af-5eb422b95aa0 None admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:46.037842 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c45350-1ed5-410f-b1af-5eb422b95aa0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:46.038351 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c45350-1ed5-410f-b1af-5eb422b95aa0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:46.039965 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c45350-1ed5-410f-b1af-5eb422b95aa0 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:46.053647 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 298/604] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Aug 10 08:23:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:46.054940 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c45350-1ed5-410f-b1af-5eb422b95aa0 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:46.059640 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97309e68-f2fb-4dc7-8fa4-6e8c22e5c2da None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:46.102769 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243fbd55-75af-4098-9152-383070acee08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:46.103523 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243fbd55-75af-4098-9152-383070acee08 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:46.104238 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243fbd55-75af-4098-9152-383070acee08 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:46.104515 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243fbd55-75af-4098-9152-383070acee08 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:46.119121 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243fbd55-75af-4098-9152-383070acee08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 08:23:46.126111 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-243fbd55-75af-4098-9152-383070acee08 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 08:23:46.126111 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:46.126111 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:46.126111 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:46.126111 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:46.126111 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:46.126111 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:46.126111 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:46.126111 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:46.126111 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:46.126111 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:46.126111 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:46.126111 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 08:23:46.126111 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 08:23:46.126111 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 08:23:46.126111 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 08:23:46.127054 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 08:23:46.127054 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 08:23:46.127054 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 08:23:46.127054 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 08:23:46.127054 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:46.127054 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:46.127054 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:46.127054 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:46.127054 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:46.127054 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:46.127054 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:46.127054 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:46.127054 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:46.127054 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:46.127054 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:46.127054 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:46.127950 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:46.127950 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:46.127950 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:46.127950 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:46.127950 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 08:23:46.127950 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 08:23:46.127950 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 08:23:46.127950 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 08:23:46.127950 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 08:23:46.127950 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 08:23:46.127950 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 08:23:46.127950 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:23:46.127950 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 307/605] 10.222.0.57 () {56 vars in 1102 bytes} [Wed Aug 10 08:23:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:46.128740 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243fbd55-75af-4098-9152-383070acee08 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:46.186553 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c45350-1ed5-410f-b1af-5eb422b95aa0 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:46.197190 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:46.216151 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:46.231452 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaae608d-937e-419c-aaba-dac4cbedfd4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:46.232324 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaae608d-937e-419c-aaba-dac4cbedfd4c None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:46.232752 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaae608d-937e-419c-aaba-dac4cbedfd4c None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:46.233203 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaae608d-937e-419c-aaba-dac4cbedfd4c None admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:46.234073 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaae608d-937e-419c-aaba-dac4cbedfd4c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:46.234525 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaae608d-937e-419c-aaba-dac4cbedfd4c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:46.236266 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaae608d-937e-419c-aaba-dac4cbedfd4c None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:46.250873 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 299/606] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Aug 10 08:23:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:46.251987 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaae608d-937e-419c-aaba-dac4cbedfd4c None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:46.275351 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243fbd55-75af-4098-9152-383070acee08 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:46.325159 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2e8c17-06dc-4045-bd17-8e86e6b132aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:46.326057 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2e8c17-06dc-4045-bd17-8e86e6b132aa None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:46.326489 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2e8c17-06dc-4045-bd17-8e86e6b132aa None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:46.326834 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2e8c17-06dc-4045-bd17-8e86e6b132aa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:46.337469 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2e8c17-06dc-4045-bd17-8e86e6b132aa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:46.339301 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2e8c17-06dc-4045-bd17-8e86e6b132aa None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:46.355755 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-de2e8c17-06dc-4045-bd17-8e86e6b132aa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 08:23:46.355755 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:46.355755 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:46.355755 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:46.355755 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:46.355755 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:46.355755 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:46.355755 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:46.355755 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:46.355755 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:46.355755 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:46.355755 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:46.355755 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 08:23:46.355755 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 08:23:46.355755 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 08:23:46.356769 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 08:23:46.356769 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:46.356769 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:46.356769 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:46.356769 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:46.356769 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:46.356769 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:46.356769 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:46.356769 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:46.356769 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:46.356769 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:46.356769 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:46.356769 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:46.356769 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:46.356769 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:46.356769 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:46.356769 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:46.357759 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 08:23:46.357759 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 08:23:46.357759 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 08:23:46.357759 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 08:23:46.357759 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 08:23:46.357759 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 08:23:46.357759 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 08:23:46.357759 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:23:46.361060 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 308/607] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Aug 10 08:23:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:46.362087 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2e8c17-06dc-4045-bd17-8e86e6b132aa None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:46.384145 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaae608d-937e-419c-aaba-dac4cbedfd4c None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:46.427601 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d17ed31-921e-4427-ae9b-04ea4e920339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:46.429103 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d17ed31-921e-4427-ae9b-04ea4e920339 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:46.429777 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d17ed31-921e-4427-ae9b-04ea4e920339 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:46.430254 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d17ed31-921e-4427-ae9b-04ea4e920339 None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:46.431214 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d17ed31-921e-4427-ae9b-04ea4e920339 None admin] RBAC: Adding query filter params (name=service) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:46.431718 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d17ed31-921e-4427-ae9b-04ea4e920339 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:46.433464 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d17ed31-921e-4427-ae9b-04ea4e920339 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:46.447954 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 300/608] 10.222.0.57 () {56 vars in 1103 bytes} [Wed Aug 10 08:23:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:46.450274 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d17ed31-921e-4427-ae9b-04ea4e920339 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:46.502187 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:46.502522 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2e8c17-06dc-4045-bd17-8e86e6b132aa None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:46.516989 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:46.549760 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c8b121-484c-4b63-b8b9-4f40a10e518d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:46.550515 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c8b121-484c-4b63-b8b9-4f40a10e518d None admin] REQUEST_METHOD: `PUT` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:46.550888 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c8b121-484c-4b63-b8b9-4f40a10e518d None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:46.551158 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c8b121-484c-4b63-b8b9-4f40a10e518d None admin] PATH_INFO: `/v3/projects/b49e381419e940278bcb6111899589b5/users/a943008193914f76a663d489bd73e873/roles/73fdd60e2f804ee69b76fa9145a472ff` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:46.568116 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d17ed31-921e-4427-ae9b-04ea4e920339 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:46.572723 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c8b121-484c-4b63-b8b9-4f40a10e518d None admin] RBAC: Authorizing `identity:create_grant(project_id=b49e381419e940278bcb6111899589b5, user_id=a943008193914f76a663d489bd73e873, role_id=73fdd60e2f804ee69b76fa9145a472ff)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:46.574227 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c8b121-484c-4b63-b8b9-4f40a10e518d None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:46.616574 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53cf556-b697-4587-9ed0-36e1c2c30ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:46.617217 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53cf556-b697-4587-9ed0-36e1c2c30ffb None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:46.617495 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53cf556-b697-4587-9ed0-36e1c2c30ffb None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:46.617760 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53cf556-b697-4587-9ed0-36e1c2c30ffb None admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:46.618499 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53cf556-b697-4587-9ed0-36e1c2c30ffb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:46.618803 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53cf556-b697-4587-9ed0-36e1c2c30ffb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:46.620471 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53cf556-b697-4587-9ed0-36e1c2c30ffb None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:46.637045 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 301/609] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Aug 10 08:23:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:46.637045 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53cf556-b697-4587-9ed0-36e1c2c30ffb None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:46.682112 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 309/610] 10.222.0.57 () {58 vars in 1439 bytes} [Wed Aug 10 08:23:46 2022] PUT /identity/v3/projects/b49e381419e940278bcb6111899589b5/users/a943008193914f76a663d489bd73e873/roles/73fdd60e2f804ee69b76fa9145a472ff => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:23:46.683246 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c8b121-484c-4b63-b8b9-4f40a10e518d None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:46.775667 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53cf556-b697-4587-9ed0-36e1c2c30ffb None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:46.779496 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:46.804878 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c8b121-484c-4b63-b8b9-4f40a10e518d None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:46.828308 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7401a0-aa32-4b35-8255-d5978b2fa615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:46.829742 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7401a0-aa32-4b35-8255-d5978b2fa615 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:46.830247 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7401a0-aa32-4b35-8255-d5978b2fa615 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:46.830965 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7401a0-aa32-4b35-8255-d5978b2fa615 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:46.845322 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7401a0-aa32-4b35-8255-d5978b2fa615 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:46.846341 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e62a10-a7ae-4def-b449-c5b3ca4d2331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:46.847107 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e62a10-a7ae-4def-b449-c5b3ca4d2331 None admin] REQUEST_METHOD: `PUT` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:46.847974 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e62a10-a7ae-4def-b449-c5b3ca4d2331 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:46.848321 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7401a0-aa32-4b35-8255-d5978b2fa615 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:46.848821 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e62a10-a7ae-4def-b449-c5b3ca4d2331 None admin] PATH_INFO: `/v3/projects/b49e381419e940278bcb6111899589b5/users/d4863ecff21e408c9deca645b88d3727/roles/73fdd60e2f804ee69b76fa9145a472ff` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:46.864619 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-ac7401a0-aa32-4b35-8255-d5978b2fa615 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:23:46.864619 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:46.864619 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:46.864619 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:46.864619 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:46.864619 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:46.864619 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:46.864619 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:46.864619 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:46.864619 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:46.864619 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:46.864619 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:46.864619 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 08:23:46.864619 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 08:23:46.864619 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 08:23:46.864619 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 08:23:46.865679 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:46.865679 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:46.865679 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:46.865679 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:46.865679 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:46.865679 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:46.865679 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:46.865679 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:46.865679 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:46.865679 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:46.865679 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:46.865679 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:46.865679 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:46.865679 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:46.865679 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:46.865679 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:46.865679 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 08:23:46.866587 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 08:23:46.866587 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:23:46.866587 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:23:46.866760 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e62a10-a7ae-4def-b449-c5b3ca4d2331 None admin] RBAC: Authorizing `identity:create_grant(project_id=b49e381419e940278bcb6111899589b5, user_id=d4863ecff21e408c9deca645b88d3727, role_id=73fdd60e2f804ee69b76fa9145a472ff)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:46.867425 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e62a10-a7ae-4def-b449-c5b3ca4d2331 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:46.867825 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 302/611] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:23:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:46.869722 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bf0628-289a-4ea2-a5c1-cfaa27082923 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:46.869722 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bf0628-289a-4ea2-a5c1-cfaa27082923 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:46.869722 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bf0628-289a-4ea2-a5c1-cfaa27082923 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:46.871101 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 303/612] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:23:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:23:46.873970 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bf0628-289a-4ea2-a5c1-cfaa27082923 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:46.939843 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 310/613] 10.222.0.57 () {58 vars in 1439 bytes} [Wed Aug 10 08:23:46 2022] PUT /identity/v3/projects/b49e381419e940278bcb6111899589b5/users/d4863ecff21e408c9deca645b88d3727/roles/73fdd60e2f804ee69b76fa9145a472ff => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:23:46.946006 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833f3d76-d1e7-478d-bdc4-15a53801e284 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:46.946432 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833f3d76-d1e7-478d-bdc4-15a53801e284 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:46.946781 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833f3d76-d1e7-478d-bdc4-15a53801e284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:46.982138 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-833f3d76-d1e7-478d-bdc4-15a53801e284 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:23:46.983502 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-833f3d76-d1e7-478d-bdc4-15a53801e284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:23:46.989505 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:46.997882 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bf0628-289a-4ea2-a5c1-cfaa27082923 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:47.059949 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a88649-ca17-42d0-a286-ba9dc7f4779b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:47.060494 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a88649-ca17-42d0-a286-ba9dc7f4779b None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:47.061238 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a88649-ca17-42d0-a286-ba9dc7f4779b None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:47.061516 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a88649-ca17-42d0-a286-ba9dc7f4779b None admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:47.062186 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a88649-ca17-42d0-a286-ba9dc7f4779b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:47.062491 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a88649-ca17-42d0-a286-ba9dc7f4779b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:47.063916 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a88649-ca17-42d0-a286-ba9dc7f4779b None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:47.093631 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 304/614] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Aug 10 08:23:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:47.099855 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a88649-ca17-42d0-a286-ba9dc7f4779b None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:47.140408 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 311/615] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:23:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 08:23:47.142352 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:47.165792 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833f3d76-d1e7-478d-bdc4-15a53801e284 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:47.256662 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a88649-ca17-42d0-a286-ba9dc7f4779b None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:47.306470 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833f3d76-d1e7-478d-bdc4-15a53801e284 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:47.313674 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc7c0bf-ae8b-4dff-8d91-866ea1266c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:47.314254 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc7c0bf-ae8b-4dff-8d91-866ea1266c97 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:47.314533 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc7c0bf-ae8b-4dff-8d91-866ea1266c97 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:47.314802 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc7c0bf-ae8b-4dff-8d91-866ea1266c97 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:47.329963 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc7c0bf-ae8b-4dff-8d91-866ea1266c97 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:47.332233 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc7c0bf-ae8b-4dff-8d91-866ea1266c97 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:47.354384 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-ccc7c0bf-ae8b-4dff-8d91-866ea1266c97 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 10 08:23:47.354384 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:47.354384 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:47.354384 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:47.354384 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:47.354384 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:47.354384 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:47.354384 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:47.354384 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:47.354384 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:47.354384 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:47.354384 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:47.354384 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 08:23:47.354384 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 08:23:47.354384 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 08:23:47.354384 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 08:23:47.355403 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:47.355403 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:47.355403 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 08:23:47.355403 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 08:23:47.355403 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 08:23:47.355403 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 08:23:47.355403 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:47.355403 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:47.355403 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:47.355403 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:47.355403 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:47.355403 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:47.355403 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:47.355403 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:47.355403 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:47.355403 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:47.355403 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:47.355403 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:47.356389 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:47.356389 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:47.356389 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 08:23:47.356389 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 08:23:47.356389 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 08:23:47.356389 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 08:23:47.356389 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 08:23:47.356389 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 08:23:47.356389 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 10 08:23:47.356389 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:23:47.356389 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 305/616] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Aug 10 08:23:47 2022] GET /identity/v3/users/swift => generated 83 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:47.358795 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef3bbc9-77e8-4799-9fba-26493f1b6fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:47.359665 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef3bbc9-77e8-4799-9fba-26493f1b6fec None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:47.360135 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef3bbc9-77e8-4799-9fba-26493f1b6fec None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:47.360563 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef3bbc9-77e8-4799-9fba-26493f1b6fec None admin] PATH_INFO: `/v3/roles/service` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:47.362154 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc7c0bf-ae8b-4dff-8d91-866ea1266c97 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:47.381421 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:47.381421 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef3bbc9-77e8-4799-9fba-26493f1b6fec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 08:23:47.381421 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-5ef3bbc9-77e8-4799-9fba-26493f1b6fec None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 08:23:47.381421 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:47.381421 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:47.381421 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:47.381421 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:47.381421 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:47.381421 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:47.381421 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:47.381421 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:47.381421 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:47.381421 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:47.382402 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:47.382402 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 08:23:47.382402 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 08:23:47.382402 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 08:23:47.382402 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 08:23:47.382402 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 08:23:47.382402 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 08:23:47.382402 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 08:23:47.382402 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 08:23:47.382402 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:47.382402 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:47.382402 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:47.382402 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:47.382402 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:47.382402 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:47.382402 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:47.382402 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:47.383555 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:47.383555 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:47.383555 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:47.383555 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:47.383555 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:47.383555 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:47.383555 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:47.383555 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:47.383555 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 08:23:47.383555 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 08:23:47.383555 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 08:23:47.383555 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 08:23:47.383555 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 08:23:47.383555 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 08:23:47.383555 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 08:23:47.383555 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:23:47.389198 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 312/617] 10.222.0.57 () {56 vars in 1102 bytes} [Wed Aug 10 08:23:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 08:23:47.413825 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef3bbc9-77e8-4799-9fba-26493f1b6fec None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:47.538429 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:47.583516 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc7c0bf-ae8b-4dff-8d91-866ea1266c97 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:47.585111 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef3bbc9-77e8-4799-9fba-26493f1b6fec None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:47.647007 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3b84a7-b251-412a-b001-cb66871b9012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:47.647624 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3b84a7-b251-412a-b001-cb66871b9012 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:47.647902 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3b84a7-b251-412a-b001-cb66871b9012 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:47.648173 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3b84a7-b251-412a-b001-cb66871b9012 None admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:47.648842 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3b84a7-b251-412a-b001-cb66871b9012 None admin] RBAC: Adding query filter params (name=service) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:47.649167 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3b84a7-b251-412a-b001-cb66871b9012 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:47.650930 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3b84a7-b251-412a-b001-cb66871b9012 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:47.652120 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2842defa-8455-4ee0-b6df-831d18218ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:47.652763 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2842defa-8455-4ee0-b6df-831d18218ef8 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:47.653065 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2842defa-8455-4ee0-b6df-831d18218ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:47.653332 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2842defa-8455-4ee0-b6df-831d18218ef8 None admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:47.654118 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2842defa-8455-4ee0-b6df-831d18218ef8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:47.654419 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2842defa-8455-4ee0-b6df-831d18218ef8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:47.655822 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2842defa-8455-4ee0-b6df-831d18218ef8 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:47.669072 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 313/618] 10.222.0.57 () {56 vars in 1103 bytes} [Wed Aug 10 08:23:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:23:47.672172 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3b84a7-b251-412a-b001-cb66871b9012 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:47.692574 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 306/619] 10.222.0.57 () {56 vars in 1135 bytes} [Wed Aug 10 08:23:47 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:47.697859 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2842defa-8455-4ee0-b6df-831d18218ef8 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:47.797571 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:47.870446 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3b84a7-b251-412a-b001-cb66871b9012 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:47.902811 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2842defa-8455-4ee0-b6df-831d18218ef8 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:47.907469 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d53f9a-2753-4744-b2c4-337da0d8e680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:47.908138 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d53f9a-2753-4744-b2c4-337da0d8e680 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:47.908420 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d53f9a-2753-4744-b2c4-337da0d8e680 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:47.908687 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d53f9a-2753-4744-b2c4-337da0d8e680 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:47.922472 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d53f9a-2753-4744-b2c4-337da0d8e680 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:47.922472 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d53f9a-2753-4744-b2c4-337da0d8e680 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:47.940598 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-b7d53f9a-2753-4744-b2c4-337da0d8e680 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:23:47.940598 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:47.940598 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:47.940598 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:47.940598 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:47.940598 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:47.940598 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:47.940598 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:47.940598 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:47.940598 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:47.940598 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:47.940598 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:47.940598 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 08:23:47.940598 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 08:23:47.940598 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 08:23:47.940598 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 08:23:47.942077 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:47.942077 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:47.942077 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:47.942077 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:47.942077 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:47.942077 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:47.942077 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:47.942077 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:47.942077 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:47.942077 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:47.942077 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:47.942077 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:47.942077 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:47.942077 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:47.942077 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:47.942077 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:47.942077 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 08:23:47.943193 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 08:23:47.943193 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:23:47.943193 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:23:47.943193 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 314/620] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:23:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 08:23:47.950008 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d53f9a-2753-4744-b2c4-337da0d8e680 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:48.005650 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be54b16-ccc0-46f5-b335-3a1155d512d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:48.006300 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be54b16-ccc0-46f5-b335-3a1155d512d1 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:48.006582 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be54b16-ccc0-46f5-b335-3a1155d512d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:48.006856 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be54b16-ccc0-46f5-b335-3a1155d512d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:48.018582 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:48.045755 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be54b16-ccc0-46f5-b335-3a1155d512d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:48.047353 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be54b16-ccc0-46f5-b335-3a1155d512d1 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:48.071128 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d53f9a-2753-4744-b2c4-337da0d8e680 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:48.074638 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-0be54b16-ccc0-46f5-b335-3a1155d512d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:23:48.074638 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:48.074638 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:48.074638 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:48.074638 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:48.074638 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:48.074638 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:48.074638 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:48.074638 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:48.074638 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:48.074638 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:48.074638 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:48.074638 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 08:23:48.074638 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 08:23:48.074638 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 08:23:48.074638 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 08:23:48.075765 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:48.075765 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:48.075765 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:48.075765 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:48.075765 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:48.075765 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:48.075765 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:48.075765 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:48.075765 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:48.075765 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:48.075765 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:48.075765 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:48.075765 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:48.075765 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:48.075765 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:48.075765 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:48.075765 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 08:23:48.076873 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 08:23:48.076873 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:23:48.076873 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:23:48.076873 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 307/621] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:23:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 378 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:48.077355 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62513deb-5471-4a06-8845-054d79b6e666 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:48.077639 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62513deb-5471-4a06-8845-054d79b6e666 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:48.077910 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62513deb-5471-4a06-8845-054d79b6e666 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:48.083907 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 308/622] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:23:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:23:48.084902 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62513deb-5471-4a06-8845-054d79b6e666 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:48.110425 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e3f1f5-3a2a-4426-b54e-45dda22c89c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:48.111137 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e3f1f5-3a2a-4426-b54e-45dda22c89c8 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:48.111417 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e3f1f5-3a2a-4426-b54e-45dda22c89c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:48.111703 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e3f1f5-3a2a-4426-b54e-45dda22c89c8 None admin] PATH_INFO: `/v3/domains` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:48.114073 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e3f1f5-3a2a-4426-b54e-45dda22c89c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:48.114403 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e3f1f5-3a2a-4426-b54e-45dda22c89c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:48.116269 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e3f1f5-3a2a-4426-b54e-45dda22c89c8 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:48.125335 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:48.135314 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 315/623] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Aug 10 08:23:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:48.138062 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ca380a-149b-43d0-b9c3-d6b50b3c191f None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:48.138062 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ca380a-149b-43d0-b9c3-d6b50b3c191f None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:48.138062 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ca380a-149b-43d0-b9c3-d6b50b3c191f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:48.163452 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-50ca380a-149b-43d0-b9c3-d6b50b3c191f None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:23:48.165348 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-50ca380a-149b-43d0-b9c3-d6b50b3c191f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:23:48.204506 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62513deb-5471-4a06-8845-054d79b6e666 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:48.241340 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fea3450-9897-45e7-99ea-d30200a37f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:48.242103 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fea3450-9897-45e7-99ea-d30200a37f3d None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:48.242608 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fea3450-9897-45e7-99ea-d30200a37f3d None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:48.242975 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fea3450-9897-45e7-99ea-d30200a37f3d None admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:48.244014 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fea3450-9897-45e7-99ea-d30200a37f3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:48.244591 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fea3450-9897-45e7-99ea-d30200a37f3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:48.246701 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fea3450-9897-45e7-99ea-d30200a37f3d None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:48.251624 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 316/624] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:23:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 08:23:48.252502 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ca380a-149b-43d0-b9c3-d6b50b3c191f None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:48.271028 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 309/625] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Aug 10 08:23:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:48.281946 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fea3450-9897-45e7-99ea-d30200a37f3d None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:48.344942 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:48.406154 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ca380a-149b-43d0-b9c3-d6b50b3c191f None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:48.409174 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:48.414464 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fea3450-9897-45e7-99ea-d30200a37f3d None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:48.453800 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfa8d21-12ea-4965-9fc0-79cfe63de3ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:48.454217 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5e6e99-d31b-4558-ad60-c7573a034554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:48.454434 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfa8d21-12ea-4965-9fc0-79cfe63de3ef None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:48.454716 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfa8d21-12ea-4965-9fc0-79cfe63de3ef None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:48.454984 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfa8d21-12ea-4965-9fc0-79cfe63de3ef None admin] PATH_INFO: `/v3/users/glance` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:48.455697 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5e6e99-d31b-4558-ad60-c7573a034554 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:48.456172 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5e6e99-d31b-4558-ad60-c7573a034554 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:48.456735 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5e6e99-d31b-4558-ad60-c7573a034554 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:48.466147 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5e6e99-d31b-4558-ad60-c7573a034554 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:48.466325 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfa8d21-12ea-4965-9fc0-79cfe63de3ef None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:48.467762 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfa8d21-12ea-4965-9fc0-79cfe63de3ef None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:48.468239 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5e6e99-d31b-4558-ad60-c7573a034554 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:48.480116 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-df5e6e99-d31b-4558-ad60-c7573a034554 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 08:23:48.480116 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:48.480116 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:48.480116 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:48.480116 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:48.480116 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:48.480116 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:48.480116 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:48.480116 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:48.480116 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:48.480116 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:48.480116 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:48.480116 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 08:23:48.480116 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 08:23:48.480116 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 08:23:48.481097 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 08:23:48.481097 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:48.481097 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:48.481097 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:48.481097 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:48.481097 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:48.481097 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:48.481097 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:48.481097 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:48.481097 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:48.481097 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:48.481097 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:48.481097 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:48.481097 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:48.481097 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:48.481097 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:48.481097 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:48.482013 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 08:23:48.482013 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 08:23:48.482013 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 08:23:48.482013 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 08:23:48.482013 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 08:23:48.482013 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 08:23:48.482013 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 08:23:48.482013 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:23:48.482471 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-bcfa8d21-12ea-4965-9fc0-79cfe63de3ef None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 10 08:23:48.482471 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:48.482471 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:48.482471 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:48.482471 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:48.482471 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:48.482471 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:48.482471 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:48.482471 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:48.482471 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:48.482471 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:48.482471 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:48.482471 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 08:23:48.482471 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 08:23:48.482471 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 08:23:48.482471 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 08:23:48.483368 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:48.483368 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:48.483368 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 08:23:48.483368 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 08:23:48.483368 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 08:23:48.483368 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 08:23:48.483368 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:48.483368 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:48.483368 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:48.483368 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:48.483368 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:48.483368 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:48.483368 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:48.483368 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:48.483368 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:48.483368 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:48.483368 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:48.483368 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:48.484350 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:48.484350 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:48.484350 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 08:23:48.484350 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 08:23:48.484350 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 08:23:48.484350 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 08:23:48.484350 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 08:23:48.484350 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 08:23:48.484350 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 10 08:23:48.484350 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:23:48.484350 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 317/626] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Aug 10 08:23:48 2022] GET /identity/v3/users/glance => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:48.484350 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfa8d21-12ea-4965-9fc0-79cfe63de3ef None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:48.486369 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 310/627] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Aug 10 08:23:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:48.491399 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5e6e99-d31b-4558-ad60-c7573a034554 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:48.608587 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfa8d21-12ea-4965-9fc0-79cfe63de3ef None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:48.611664 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5e6e99-d31b-4558-ad60-c7573a034554 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:48.621182 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:48.649044 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d6ac1c-f645-4f1e-a1cc-94a22df0cdf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:48.649654 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d6ac1c-f645-4f1e-a1cc-94a22df0cdf8 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:48.650059 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d6ac1c-f645-4f1e-a1cc-94a22df0cdf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:48.650256 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d6ac1c-f645-4f1e-a1cc-94a22df0cdf8 None admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:48.651006 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d6ac1c-f645-4f1e-a1cc-94a22df0cdf8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:48.651335 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d6ac1c-f645-4f1e-a1cc-94a22df0cdf8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:48.652810 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d6ac1c-f645-4f1e-a1cc-94a22df0cdf8 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:48.657780 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd8bd73-c084-498b-8b84-3b77ac98579c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:48.658743 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd8bd73-c084-498b-8b84-3b77ac98579c None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:48.659226 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd8bd73-c084-498b-8b84-3b77ac98579c None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:48.660130 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd8bd73-c084-498b-8b84-3b77ac98579c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:48.673884 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 311/628] 10.222.0.57 () {56 vars in 1137 bytes} [Wed Aug 10 08:23:48 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:48.674879 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d6ac1c-f645-4f1e-a1cc-94a22df0cdf8 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:48.676950 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd8bd73-c084-498b-8b84-3b77ac98579c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 08:23:48.683697 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-7dd8bd73-c084-498b-8b84-3b77ac98579c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 08:23:48.683697 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:48.683697 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:48.683697 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:48.683697 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:48.683697 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:48.683697 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:48.683697 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:48.683697 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:48.683697 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:48.683697 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:48.683697 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:48.683697 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 08:23:48.683697 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 08:23:48.683697 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 08:23:48.683697 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 08:23:48.684626 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 08:23:48.684626 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 08:23:48.684626 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 08:23:48.684626 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 08:23:48.684626 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:48.684626 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:48.684626 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:48.684626 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:48.684626 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:48.684626 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:48.684626 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:48.684626 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:48.684626 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:48.684626 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:48.684626 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:48.684626 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:48.685528 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:48.685528 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:48.685528 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:48.685528 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:48.685528 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 08:23:48.685528 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 08:23:48.685528 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 08:23:48.685528 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 08:23:48.685528 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 08:23:48.685528 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 08:23:48.685528 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 08:23:48.685528 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:23:48.692627 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 318/629] 10.222.0.57 () {56 vars in 1102 bytes} [Wed Aug 10 08:23:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:48.692627 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd8bd73-c084-498b-8b84-3b77ac98579c None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:48.727108 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:48.790816 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d6ac1c-f645-4f1e-a1cc-94a22df0cdf8 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:48.812252 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd8bd73-c084-498b-8b84-3b77ac98579c None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:48.827548 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48252deb-aff1-445c-9611-4e00f06b9d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:48.827950 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48252deb-aff1-445c-9611-4e00f06b9d2e None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:48.828234 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48252deb-aff1-445c-9611-4e00f06b9d2e None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:48.828503 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48252deb-aff1-445c-9611-4e00f06b9d2e None admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:48.829265 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48252deb-aff1-445c-9611-4e00f06b9d2e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:48.829560 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48252deb-aff1-445c-9611-4e00f06b9d2e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:48.830976 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48252deb-aff1-445c-9611-4e00f06b9d2e None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:48.849423 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 312/630] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Aug 10 08:23:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:48.850438 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48252deb-aff1-445c-9611-4e00f06b9d2e None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:48.853472 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472b03a5-87c2-4184-a35e-9b88d010e644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:48.854421 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472b03a5-87c2-4184-a35e-9b88d010e644 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:48.854900 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472b03a5-87c2-4184-a35e-9b88d010e644 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:48.855408 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472b03a5-87c2-4184-a35e-9b88d010e644 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:48.879349 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:48.928360 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472b03a5-87c2-4184-a35e-9b88d010e644 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:48.929842 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472b03a5-87c2-4184-a35e-9b88d010e644 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:48.955747 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:48.963162 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48252deb-aff1-445c-9611-4e00f06b9d2e None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:48.997623 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777f65df-9959-4754-80fa-cc2ef4f7cd1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:48.998233 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777f65df-9959-4754-80fa-cc2ef4f7cd1b None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:48.998568 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777f65df-9959-4754-80fa-cc2ef4f7cd1b None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:48.998850 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777f65df-9959-4754-80fa-cc2ef4f7cd1b None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:48.999548 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777f65df-9959-4754-80fa-cc2ef4f7cd1b None admin] RBAC: Adding query filter params (name=service) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:48.999852 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777f65df-9959-4754-80fa-cc2ef4f7cd1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:49.001241 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777f65df-9959-4754-80fa-cc2ef4f7cd1b None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:49.009690 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 313/631] 10.222.0.57 () {56 vars in 1103 bytes} [Wed Aug 10 08:23:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:49.010460 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777f65df-9959-4754-80fa-cc2ef4f7cd1b None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:49.048370 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 319/632] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Aug 10 08:23:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 08:23:49.049625 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84429248-f97f-4675-848a-c7be97aef0e1 None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:49.049980 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84429248-f97f-4675-848a-c7be97aef0e1 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:49.050342 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84429248-f97f-4675-848a-c7be97aef0e1 None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:49.052018 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 320/633] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:23:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:23:49.052604 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84429248-f97f-4675-848a-c7be97aef0e1 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:49.109456 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:49.125048 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777f65df-9959-4754-80fa-cc2ef4f7cd1b None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:49.160600 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f94f382-a8fd-4578-8dae-a657a665e5d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:49.161315 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f94f382-a8fd-4578-8dae-a657a665e5d0 None admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:49.161596 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f94f382-a8fd-4578-8dae-a657a665e5d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:49.161869 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f94f382-a8fd-4578-8dae-a657a665e5d0 None admin] PATH_INFO: `/v3/projects/b49e381419e940278bcb6111899589b5/users/b4b26ba3a9e04c38b52db2a46198a53c/roles/73fdd60e2f804ee69b76fa9145a472ff` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:49.167559 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84429248-f97f-4675-848a-c7be97aef0e1 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:49.177201 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f94f382-a8fd-4578-8dae-a657a665e5d0 None admin] RBAC: Authorizing `identity:create_grant(project_id=b49e381419e940278bcb6111899589b5, user_id=b4b26ba3a9e04c38b52db2a46198a53c, role_id=73fdd60e2f804ee69b76fa9145a472ff)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:49.178694 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f94f382-a8fd-4578-8dae-a657a665e5d0 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:49.193706 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:49.204033 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506cd967-2bac-49fd-84fb-4c151ae0a3e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:49.204650 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506cd967-2bac-49fd-84fb-4c151ae0a3e4 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:49.204941 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506cd967-2bac-49fd-84fb-4c151ae0a3e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:49.205220 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506cd967-2bac-49fd-84fb-4c151ae0a3e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:49.216182 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506cd967-2bac-49fd-84fb-4c151ae0a3e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:49.217695 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506cd967-2bac-49fd-84fb-4c151ae0a3e4 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:49.232848 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-506cd967-2bac-49fd-84fb-4c151ae0a3e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:23:49.232848 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:49.232848 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:49.232848 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:49.232848 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:49.232848 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:49.232848 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:49.232848 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:49.232848 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:49.232848 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:49.232848 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:49.232848 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:49.232848 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 08:23:49.232848 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 08:23:49.232848 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 08:23:49.232848 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 08:23:49.237225 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:49.237225 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:49.237225 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:49.237225 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:49.237225 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:49.237225 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:49.237225 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:49.237225 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:49.237225 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:49.237225 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:49.237225 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:49.237225 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:49.237225 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:49.237225 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:49.237225 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:49.237225 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:49.237225 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 08:23:49.244294 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 08:23:49.244294 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:23:49.244294 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:23:49.244294 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 321/634] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:23:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:49.244294 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506cd967-2bac-49fd-84fb-4c151ae0a3e4 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:49.244608 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 314/635] 10.222.0.57 () {58 vars in 1439 bytes} [Wed Aug 10 08:23:49 2022] PUT /identity/v3/projects/b49e381419e940278bcb6111899589b5/users/b4b26ba3a9e04c38b52db2a46198a53c/roles/73fdd60e2f804ee69b76fa9145a472ff => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:23:49.254968 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d956dab7-807d-4ba7-b433-f6a8175648d8 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:49.255315 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d956dab7-807d-4ba7-b433-f6a8175648d8 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:49.255519 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d956dab7-807d-4ba7-b433-f6a8175648d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:49.284496 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-d956dab7-807d-4ba7-b433-f6a8175648d8 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:23:49.287006 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-d956dab7-807d-4ba7-b433-f6a8175648d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:23:49.361636 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506cd967-2bac-49fd-84fb-4c151ae0a3e4 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:49.365187 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 315/636] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:23:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 08:23:49.365719 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d956dab7-807d-4ba7-b433-f6a8175648d8 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:49.422923 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7826187b-0c69-4b20-94c6-9d447d6f1a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:49.423573 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7826187b-0c69-4b20-94c6-9d447d6f1a3c None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:49.423860 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7826187b-0c69-4b20-94c6-9d447d6f1a3c None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:49.424210 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7826187b-0c69-4b20-94c6-9d447d6f1a3c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:49.436449 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7826187b-0c69-4b20-94c6-9d447d6f1a3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:49.437861 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7826187b-0c69-4b20-94c6-9d447d6f1a3c None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:49.447908 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:49.452785 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-7826187b-0c69-4b20-94c6-9d447d6f1a3c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:23:49.452785 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:49.452785 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:49.452785 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:49.452785 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:49.452785 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:49.452785 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:49.452785 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:49.452785 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:49.452785 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:49.452785 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:49.452785 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:49.452785 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 08:23:49.452785 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 08:23:49.452785 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 08:23:49.452785 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 08:23:49.453930 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:49.453930 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:49.453930 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:49.453930 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:49.453930 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:49.453930 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:49.453930 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:49.453930 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:49.453930 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:49.453930 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:49.453930 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:49.453930 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:49.453930 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:49.453930 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:49.453930 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:49.453930 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:49.453930 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 08:23:49.455900 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 08:23:49.455900 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:23:49.455900 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:23:49.455900 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 322/637] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:23:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:49.455900 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495e4181-e30f-481e-a0c9-1c9b1947db94 None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:49.455900 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495e4181-e30f-481e-a0c9-1c9b1947db94 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:49.455900 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495e4181-e30f-481e-a0c9-1c9b1947db94 None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:49.457867 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 323/638] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:23:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:23:49.458588 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495e4181-e30f-481e-a0c9-1c9b1947db94 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:49.491315 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d956dab7-807d-4ba7-b433-f6a8175648d8 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:49.491666 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:49.526793 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8d2118-409a-489a-a94d-327b2bdbe69b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:49.527401 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8d2118-409a-489a-a94d-327b2bdbe69b None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:49.527679 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8d2118-409a-489a-a94d-327b2bdbe69b None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:49.527953 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8d2118-409a-489a-a94d-327b2bdbe69b None admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:49.528591 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8d2118-409a-489a-a94d-327b2bdbe69b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:49.528892 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8d2118-409a-489a-a94d-327b2bdbe69b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:49.530274 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8d2118-409a-489a-a94d-327b2bdbe69b None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:49.546567 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 316/639] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Aug 10 08:23:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:49.546567 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29ec36c-b686-4f0f-bf3a-7a13db14fc0f None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:49.546567 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29ec36c-b686-4f0f-bf3a-7a13db14fc0f None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:49.546567 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29ec36c-b686-4f0f-bf3a-7a13db14fc0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:49.564829 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-c29ec36c-b686-4f0f-bf3a-7a13db14fc0f None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:23:49.566135 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-c29ec36c-b686-4f0f-bf3a-7a13db14fc0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:23:49.580296 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495e4181-e30f-481e-a0c9-1c9b1947db94 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:49.615067 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2612d3c2-7918-4e8d-9795-fa447e8d7fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:49.615694 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2612d3c2-7918-4e8d-9795-fa447e8d7fdd None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:49.615973 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2612d3c2-7918-4e8d-9795-fa447e8d7fdd None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:49.616239 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2612d3c2-7918-4e8d-9795-fa447e8d7fdd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:49.627651 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2612d3c2-7918-4e8d-9795-fa447e8d7fdd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 08:23:49.633645 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-2612d3c2-7918-4e8d-9795-fa447e8d7fdd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 08:23:49.633645 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:49.633645 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:49.633645 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:49.633645 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:49.633645 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:49.633645 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:49.633645 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:49.633645 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:49.633645 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:49.633645 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:49.633645 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:49.633645 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 08:23:49.633645 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 08:23:49.633645 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 08:23:49.633645 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 08:23:49.634789 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 08:23:49.634789 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 08:23:49.634789 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 08:23:49.634789 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 08:23:49.634789 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:49.634789 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:49.634789 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:49.634789 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:49.634789 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:49.634789 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:49.634789 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:49.634789 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:49.634789 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:49.634789 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:49.634789 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:49.634789 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:49.635839 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:49.635839 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:49.635839 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:49.635839 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:49.635839 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 08:23:49.635839 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 08:23:49.635839 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 08:23:49.635839 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 08:23:49.635839 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 08:23:49.635839 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 08:23:49.635839 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 08:23:49.635839 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:23:49.639474 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 324/640] 10.222.0.57 () {56 vars in 1102 bytes} [Wed Aug 10 08:23:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:49.640368 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2612d3c2-7918-4e8d-9795-fa447e8d7fdd None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:49.644724 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 317/641] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:23:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 08:23:49.646171 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29ec36c-b686-4f0f-bf3a-7a13db14fc0f None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:49.717252 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:49.734848 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:49.748484 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2612d3c2-7918-4e8d-9795-fa447e8d7fdd None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:49.765069 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29ec36c-b686-4f0f-bf3a-7a13db14fc0f None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:49.783921 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccd00a1-93d0-48ab-8de9-8f83f067e19d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:49.784530 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccd00a1-93d0-48ab-8de9-8f83f067e19d None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:49.785521 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccd00a1-93d0-48ab-8de9-8f83f067e19d None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:49.785802 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccd00a1-93d0-48ab-8de9-8f83f067e19d None admin] PATH_INFO: `/v3/domains` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:49.786456 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccd00a1-93d0-48ab-8de9-8f83f067e19d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:49.786756 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccd00a1-93d0-48ab-8de9-8f83f067e19d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:49.788131 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccd00a1-93d0-48ab-8de9-8f83f067e19d None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:49.806198 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 325/642] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Aug 10 08:23:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:49.806371 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df33fcc-396a-41f1-8a22-b319fe02baa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:49.806371 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df33fcc-396a-41f1-8a22-b319fe02baa1 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:49.806585 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccd00a1-93d0-48ab-8de9-8f83f067e19d None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:49.806683 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df33fcc-396a-41f1-8a22-b319fe02baa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:49.806683 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df33fcc-396a-41f1-8a22-b319fe02baa1 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:49.814469 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df33fcc-396a-41f1-8a22-b319fe02baa1 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:49.816027 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df33fcc-396a-41f1-8a22-b319fe02baa1 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:49.828862 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-7df33fcc-396a-41f1-8a22-b319fe02baa1 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 10 08:23:49.828862 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:49.828862 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:49.828862 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:49.828862 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:49.828862 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:49.828862 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:49.828862 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:49.828862 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:49.828862 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:49.828862 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:49.828862 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:49.828862 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 08:23:49.828862 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 08:23:49.828862 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 08:23:49.828862 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 08:23:49.830052 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:49.830052 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:49.830052 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 08:23:49.830052 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 08:23:49.830052 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 08:23:49.830052 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 08:23:49.830052 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:49.830052 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:49.830052 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:49.830052 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:49.830052 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:49.830052 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:49.830052 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:49.830052 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:49.830052 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:49.830052 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:49.830052 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:49.830052 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:49.831209 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:49.831209 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:49.831209 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 08:23:49.831209 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 08:23:49.831209 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 08:23:49.831209 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 08:23:49.831209 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 08:23:49.831209 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 08:23:49.831209 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 10 08:23:49.831209 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:23:49.838009 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 318/643] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Aug 10 08:23:49 2022] GET /identity/v3/users/cinder => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:49.838977 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df33fcc-396a-41f1-8a22-b319fe02baa1 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:49.907357 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccd00a1-93d0-48ab-8de9-8f83f067e19d None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:49.944888 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f854ff6-58eb-4018-9a9a-c459963bb698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:49.945664 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df33fcc-396a-41f1-8a22-b319fe02baa1 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:49.945781 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f854ff6-58eb-4018-9a9a-c459963bb698 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:49.946119 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f854ff6-58eb-4018-9a9a-c459963bb698 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:49.946486 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f854ff6-58eb-4018-9a9a-c459963bb698 None admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:49.947241 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f854ff6-58eb-4018-9a9a-c459963bb698 None admin] RBAC: Adding query filter params (name=service) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:49.947642 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f854ff6-58eb-4018-9a9a-c459963bb698 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:49.949142 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f854ff6-58eb-4018-9a9a-c459963bb698 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:49.962273 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 326/644] 10.222.0.57 () {56 vars in 1103 bytes} [Wed Aug 10 08:23:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:49.962273 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f854ff6-58eb-4018-9a9a-c459963bb698 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:49.966757 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:49.976170 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:49.987158 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87913efc-ab77-4f9e-b62d-7a0073723be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:49.989185 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87913efc-ab77-4f9e-b62d-7a0073723be5 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:49.989469 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87913efc-ab77-4f9e-b62d-7a0073723be5 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:49.989933 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87913efc-ab77-4f9e-b62d-7a0073723be5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:50.001212 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87913efc-ab77-4f9e-b62d-7a0073723be5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 08:23:50.007304 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-87913efc-ab77-4f9e-b62d-7a0073723be5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 08:23:50.007304 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:50.007304 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:50.007304 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:50.007304 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:50.007304 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:50.007304 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:50.007304 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:50.007304 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:50.007304 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:50.007304 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:50.007304 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:50.007304 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 08:23:50.007304 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 08:23:50.007304 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 08:23:50.007304 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 08:23:50.008258 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 08:23:50.008258 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 08:23:50.008258 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 08:23:50.008258 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 08:23:50.008258 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:50.008258 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:50.008258 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:50.008258 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:50.008258 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:50.008258 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:50.008258 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:50.008258 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:50.008258 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:50.008258 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:50.008258 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:50.008258 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:50.009860 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:50.009860 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:50.009860 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:50.009860 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:50.009860 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 08:23:50.009860 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 08:23:50.009860 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 08:23:50.009860 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 08:23:50.009860 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 08:23:50.009860 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 08:23:50.009860 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 08:23:50.009860 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:23:50.009860 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 319/645] 10.222.0.57 () {56 vars in 1102 bytes} [Wed Aug 10 08:23:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:50.009860 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87913efc-ab77-4f9e-b62d-7a0073723be5 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:50.079678 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f854ff6-58eb-4018-9a9a-c459963bb698 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:50.114624 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87913efc-ab77-4f9e-b62d-7a0073723be5 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:50.116023 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8190f55a-1944-44a9-8d40-2acf22f12c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:50.116645 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8190f55a-1944-44a9-8d40-2acf22f12c76 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:50.116944 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8190f55a-1944-44a9-8d40-2acf22f12c76 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:50.117271 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8190f55a-1944-44a9-8d40-2acf22f12c76 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:50.124331 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8190f55a-1944-44a9-8d40-2acf22f12c76 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:50.125824 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8190f55a-1944-44a9-8d40-2acf22f12c76 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:50.136854 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-8190f55a-1944-44a9-8d40-2acf22f12c76 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 08:23:50.136854 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:50.136854 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:50.136854 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:50.136854 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:50.136854 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:50.136854 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:50.136854 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:50.136854 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:50.136854 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:50.136854 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:50.136854 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:50.136854 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 08:23:50.136854 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 08:23:50.136854 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 08:23:50.137803 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 08:23:50.137803 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:50.137803 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:50.137803 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:50.137803 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:50.137803 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:50.137803 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:50.137803 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:50.137803 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:50.137803 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:50.137803 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:50.137803 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:50.137803 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:50.137803 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:50.137803 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:50.137803 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:50.137803 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:50.138749 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 08:23:50.138749 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 08:23:50.138749 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 08:23:50.138749 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 08:23:50.138749 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 08:23:50.138749 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 08:23:50.138749 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 08:23:50.138749 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:23:50.138749 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 327/646] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Aug 10 08:23:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:50.138749 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8190f55a-1944-44a9-8d40-2acf22f12c76 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:50.155483 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b6b417-5ab1-422a-96b7-51c951f57285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:50.156089 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b6b417-5ab1-422a-96b7-51c951f57285 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:50.156368 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b6b417-5ab1-422a-96b7-51c951f57285 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:50.156651 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b6b417-5ab1-422a-96b7-51c951f57285 None admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:50.157403 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b6b417-5ab1-422a-96b7-51c951f57285 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:50.157708 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b6b417-5ab1-422a-96b7-51c951f57285 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:50.159137 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b6b417-5ab1-422a-96b7-51c951f57285 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:50.181211 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 320/647] 10.222.0.57 () {56 vars in 1137 bytes} [Wed Aug 10 08:23:50 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:50.181211 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b6b417-5ab1-422a-96b7-51c951f57285 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:50.213814 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:50.218664 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:50.250192 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8190f55a-1944-44a9-8d40-2acf22f12c76 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:50.285438 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b6b417-5ab1-422a-96b7-51c951f57285 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:50.287547 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe3fae5-9cbf-4f48-8acc-73736cd88707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:50.289125 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe3fae5-9cbf-4f48-8acc-73736cd88707 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:50.289409 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe3fae5-9cbf-4f48-8acc-73736cd88707 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:50.289682 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe3fae5-9cbf-4f48-8acc-73736cd88707 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:50.303762 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe3fae5-9cbf-4f48-8acc-73736cd88707 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:50.305435 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe3fae5-9cbf-4f48-8acc-73736cd88707 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:50.321320 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-efe3fae5-9cbf-4f48-8acc-73736cd88707 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:23:50.321320 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:50.321320 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:50.321320 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:50.321320 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:50.321320 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:50.321320 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:50.321320 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:50.321320 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:50.321320 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:50.321320 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:50.321320 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:50.321320 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 08:23:50.321320 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 08:23:50.321320 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 08:23:50.321320 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 08:23:50.322523 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:50.322523 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:50.322523 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:50.322523 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:50.322523 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:50.322523 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:50.322523 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:50.322523 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:50.322523 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:50.322523 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:50.322523 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:50.322523 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:50.322523 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:50.322523 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:50.322523 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:50.322523 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:50.322523 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 08:23:50.328427 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 08:23:50.328427 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:23:50.328427 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:23:50.328427 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 328/648] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:23:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:50.328427 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe3fae5-9cbf-4f48-8acc-73736cd88707 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:50.332034 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58911a36-46f0-4eb4-a37b-9454a275140c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:50.332726 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58911a36-46f0-4eb4-a37b-9454a275140c None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:50.333110 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58911a36-46f0-4eb4-a37b-9454a275140c None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:50.333448 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58911a36-46f0-4eb4-a37b-9454a275140c None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:50.334184 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58911a36-46f0-4eb4-a37b-9454a275140c None admin] RBAC: Adding query filter params (name=service) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:50.334555 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58911a36-46f0-4eb4-a37b-9454a275140c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:50.335986 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58911a36-46f0-4eb4-a37b-9454a275140c None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:50.346536 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 321/649] 10.222.0.57 () {56 vars in 1103 bytes} [Wed Aug 10 08:23:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:50.351417 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58911a36-46f0-4eb4-a37b-9454a275140c None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:50.452684 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe3fae5-9cbf-4f48-8acc-73736cd88707 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:50.467871 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:50.474407 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58911a36-46f0-4eb4-a37b-9454a275140c None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:50.478526 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:50.490931 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797dfe53-b570-45d2-8757-cf275684d7f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:50.491532 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797dfe53-b570-45d2-8757-cf275684d7f3 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:50.491811 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797dfe53-b570-45d2-8757-cf275684d7f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:50.492077 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797dfe53-b570-45d2-8757-cf275684d7f3 None admin] PATH_INFO: `/v3/projects` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:50.492761 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797dfe53-b570-45d2-8757-cf275684d7f3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:50.493150 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797dfe53-b570-45d2-8757-cf275684d7f3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:50.494508 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797dfe53-b570-45d2-8757-cf275684d7f3 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:50.512701 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 329/650] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Aug 10 08:23:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:50.512701 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797dfe53-b570-45d2-8757-cf275684d7f3 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:50.515105 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34f9ed3-58de-4970-9797-7fea9c33a608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:50.515778 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34f9ed3-58de-4970-9797-7fea9c33a608 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:50.516132 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34f9ed3-58de-4970-9797-7fea9c33a608 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:50.516478 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34f9ed3-58de-4970-9797-7fea9c33a608 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:50.585345 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34f9ed3-58de-4970-9797-7fea9c33a608 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:50.586866 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34f9ed3-58de-4970-9797-7fea9c33a608 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:50.617734 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797dfe53-b570-45d2-8757-cf275684d7f3 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:50.655958 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629c6404-0524-41ff-89e9-58c23a53d71c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:50.656809 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629c6404-0524-41ff-89e9-58c23a53d71c None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:50.657291 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629c6404-0524-41ff-89e9-58c23a53d71c None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:50.657732 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629c6404-0524-41ff-89e9-58c23a53d71c None admin] PATH_INFO: `/v3/domains` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:50.658527 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629c6404-0524-41ff-89e9-58c23a53d71c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:50.658994 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629c6404-0524-41ff-89e9-58c23a53d71c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:50.660652 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629c6404-0524-41ff-89e9-58c23a53d71c None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:50.676775 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 330/651] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Aug 10 08:23:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:50.676775 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629c6404-0524-41ff-89e9-58c23a53d71c None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:50.694871 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 322/652] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Aug 10 08:23:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 08:23:50.694871 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34f9ed3-58de-4970-9797-7fea9c33a608 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:50.783084 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629c6404-0524-41ff-89e9-58c23a53d71c None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:50.789378 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34f9ed3-58de-4970-9797-7fea9c33a608 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:50.798991 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:50.818454 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ff1945-4004-403d-b204-82b39505819c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:50.819110 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ff1945-4004-403d-b204-82b39505819c None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:50.819392 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ff1945-4004-403d-b204-82b39505819c None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:50.819676 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ff1945-4004-403d-b204-82b39505819c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:50.827260 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca9e735-b2f3-4f18-b927-08ca3faae5c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:50.829157 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:50.829570 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca9e735-b2f3-4f18-b927-08ca3faae5c9 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:50.834842 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca9e735-b2f3-4f18-b927-08ca3faae5c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:50.834842 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca9e735-b2f3-4f18-b927-08ca3faae5c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:50.834842 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca9e735-b2f3-4f18-b927-08ca3faae5c9 None admin] RBAC: Adding query filter params (role.id=73fdd60e2f804ee69b76fa9145a472ff, scope.project.id=b49e381419e940278bcb6111899589b5, user.id=3263af99fc2f48e79fd1e5b8a1e555a4) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:50.834842 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca9e735-b2f3-4f18-b927-08ca3faae5c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:50.835166 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ff1945-4004-403d-b204-82b39505819c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:50.835249 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca9e735-b2f3-4f18-b927-08ca3faae5c9 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:50.835321 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ff1945-4004-403d-b204-82b39505819c None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:50.849306 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 323/653] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Aug 10 08:23:50 2022] GET /identity/v3/role_assignments?user.id=3263af99fc2f48e79fd1e5b8a1e555a4&scope.project.id=b49e381419e940278bcb6111899589b5&role.id=73fdd60e2f804ee69b76fa9145a472ff => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:50.849306 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca9e735-b2f3-4f18-b927-08ca3faae5c9 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:50.859417 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-27ff1945-4004-403d-b204-82b39505819c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:23:50.859417 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:50.859417 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:50.859417 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:50.859417 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:50.859417 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:50.859417 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:50.859417 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:50.859417 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:50.859417 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:50.859417 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:50.859417 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:50.859417 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 08:23:50.859417 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 08:23:50.859417 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 08:23:50.859417 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 08:23:50.860712 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:50.860712 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:50.860712 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:50.860712 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:50.860712 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:50.860712 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:50.860712 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:50.860712 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:50.860712 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:50.860712 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:50.860712 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:50.860712 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:50.860712 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:50.860712 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:50.860712 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:50.860712 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:50.860712 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 08:23:50.862042 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 08:23:50.862042 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:23:50.862042 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:23:50.862633 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 331/654] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:23:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:50.868004 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ff1945-4004-403d-b204-82b39505819c None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:50.935146 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:50.992648 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:50.996479 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca9e735-b2f3-4f18-b927-08ca3faae5c9 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:51.007126 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ff1945-4004-403d-b204-82b39505819c None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:51.048842 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0a4baf-3171-450e-b43a-0ca809ee727c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:51.050875 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0a4baf-3171-450e-b43a-0ca809ee727c None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:51.050875 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0a4baf-3171-450e-b43a-0ca809ee727c None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:51.050875 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0a4baf-3171-450e-b43a-0ca809ee727c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:51.052323 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3412293-db94-4195-ba17-171e97a8500d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:51.053823 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3412293-db94-4195-ba17-171e97a8500d None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:51.053823 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3412293-db94-4195-ba17-171e97a8500d None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:51.054034 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3412293-db94-4195-ba17-171e97a8500d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:51.062680 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3412293-db94-4195-ba17-171e97a8500d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:51.063700 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0a4baf-3171-450e-b43a-0ca809ee727c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:51.064298 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3412293-db94-4195-ba17-171e97a8500d None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:51.065370 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0a4baf-3171-450e-b43a-0ca809ee727c None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:51.079349 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-c3412293-db94-4195-ba17-171e97a8500d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 08:23:51.079349 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:51.079349 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:51.079349 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:51.079349 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:51.079349 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:51.079349 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:51.079349 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:51.079349 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:51.079349 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:51.079349 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:51.079349 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:51.079349 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 08:23:51.079349 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 08:23:51.079349 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 08:23:51.079349 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 08:23:51.080369 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:51.080369 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:51.080369 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 08:23:51.080369 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 08:23:51.080369 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 08:23:51.080369 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 08:23:51.080369 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:51.080369 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:51.080369 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:51.080369 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:51.080369 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:51.080369 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:51.080369 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:51.080369 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:51.080369 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:51.080369 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:51.080369 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:51.080369 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:51.085040 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:51.085040 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:51.085040 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 08:23:51.085040 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 08:23:51.085040 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 08:23:51.085040 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 08:23:51.085040 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 08:23:51.085040 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 08:23:51.085040 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 08:23:51.085040 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:23:51.085040 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 324/655] 10.222.0.57 () {56 vars in 1093 bytes} [Wed Aug 10 08:23:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:51.085040 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3412293-db94-4195-ba17-171e97a8500d None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:51.085777 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-0e0a4baf-3171-450e-b43a-0ca809ee727c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:23:51.085777 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:51.085777 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:51.085777 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:51.085777 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:51.085777 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:51.085777 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:51.085777 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:51.085777 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:51.085777 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:51.085777 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:51.085777 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:51.085777 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 08:23:51.085777 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 08:23:51.085777 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 08:23:51.085777 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 08:23:51.087050 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:51.087050 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:51.087050 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:51.087050 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:51.087050 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:51.087050 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:51.087050 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:51.087050 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:51.087050 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:51.087050 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:51.087050 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:51.087050 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:51.087050 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:51.087050 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:51.087050 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:51.087050 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:51.087050 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 08:23:51.087950 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 08:23:51.087950 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:23:51.087950 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:23:51.087950 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 332/656] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:23:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:51.091443 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0a4baf-3171-450e-b43a-0ca809ee727c None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:51.216027 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0a4baf-3171-450e-b43a-0ca809ee727c None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:51.216555 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3412293-db94-4195-ba17-171e97a8500d None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:51.240878 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:51.255639 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc43894-668f-486a-b09b-2c5b9b6af51b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:51.255923 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83137aa4-d6d6-4e16-9378-f31fa6dc3fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:51.258281 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83137aa4-d6d6-4e16-9378-f31fa6dc3fc8 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:51.258390 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc43894-668f-486a-b09b-2c5b9b6af51b None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:51.258492 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83137aa4-d6d6-4e16-9378-f31fa6dc3fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:51.258589 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc43894-668f-486a-b09b-2c5b9b6af51b None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:51.258682 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83137aa4-d6d6-4e16-9378-f31fa6dc3fc8 None admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:51.258782 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc43894-668f-486a-b09b-2c5b9b6af51b None admin] PATH_INFO: `/v3/domains` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:51.258869 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83137aa4-d6d6-4e16-9378-f31fa6dc3fc8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:51.258961 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc43894-668f-486a-b09b-2c5b9b6af51b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:51.259056 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83137aa4-d6d6-4e16-9378-f31fa6dc3fc8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:51.259205 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc43894-668f-486a-b09b-2c5b9b6af51b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:51.260448 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83137aa4-d6d6-4e16-9378-f31fa6dc3fc8 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:51.260884 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc43894-668f-486a-b09b-2c5b9b6af51b None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:51.277870 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 325/657] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Aug 10 08:23:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:51.277870 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83137aa4-d6d6-4e16-9378-f31fa6dc3fc8 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:51.278166 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 333/658] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Aug 10 08:23:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:23:51.279620 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc43894-668f-486a-b09b-2c5b9b6af51b None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:51.291387 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:51.391424 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83137aa4-d6d6-4e16-9378-f31fa6dc3fc8 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:51.392345 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc43894-668f-486a-b09b-2c5b9b6af51b None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:51.430443 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ae86e2-10b8-4bdd-be34-3fd82a809249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:51.431045 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ae86e2-10b8-4bdd-be34-3fd82a809249 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:51.431318 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ae86e2-10b8-4bdd-be34-3fd82a809249 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:51.431584 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ae86e2-10b8-4bdd-be34-3fd82a809249 None admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:51.432306 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ae86e2-10b8-4bdd-be34-3fd82a809249 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:51.432419 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6402fb6-239e-4ddc-afa6-8c5fa046f55d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:51.432631 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ae86e2-10b8-4bdd-be34-3fd82a809249 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:51.433216 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6402fb6-239e-4ddc-afa6-8c5fa046f55d None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:51.433501 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6402fb6-239e-4ddc-afa6-8c5fa046f55d None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:51.433778 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6402fb6-239e-4ddc-afa6-8c5fa046f55d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:51.434154 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ae86e2-10b8-4bdd-be34-3fd82a809249 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:51.442512 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6402fb6-239e-4ddc-afa6-8c5fa046f55d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:51.444063 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6402fb6-239e-4ddc-afa6-8c5fa046f55d None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:51.452741 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 326/659] 10.222.0.57 () {56 vars in 1133 bytes} [Wed Aug 10 08:23:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:51.457315 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ae86e2-10b8-4bdd-be34-3fd82a809249 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:51.457448 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-f6402fb6-239e-4ddc-afa6-8c5fa046f55d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 10 08:23:51.457448 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:51.457448 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:51.457448 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:51.457448 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:51.457448 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:51.457448 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:51.457448 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:51.457448 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:51.457448 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:51.457448 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:51.457448 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:51.457448 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 08:23:51.457448 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 08:23:51.457448 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 08:23:51.457448 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 08:23:51.458684 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:51.458684 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:51.458684 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 08:23:51.458684 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 08:23:51.458684 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 08:23:51.458684 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 08:23:51.458684 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:51.458684 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:51.458684 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:51.458684 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:51.458684 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:51.458684 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:51.458684 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:51.458684 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:51.458684 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:51.458684 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:51.458684 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:51.458684 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:51.459654 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:51.459654 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:51.459654 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 08:23:51.459654 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 08:23:51.459654 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 08:23:51.459654 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 08:23:51.459654 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 08:23:51.459654 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 08:23:51.459654 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 10 08:23:51.459654 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:23:51.459654 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 334/660] 10.222.0.57 () {56 vars in 1102 bytes} [Wed Aug 10 08:23:51 2022] GET /identity/v3/users/neutron => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:51.459654 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1911378-3c97-47b0-a839-33eb221ac2eb None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:51.459654 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1911378-3c97-47b0-a839-33eb221ac2eb None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:51.460453 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1911378-3c97-47b0-a839-33eb221ac2eb None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:51.461158 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 335/661] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:23:51 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:23:51.465599 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1911378-3c97-47b0-a839-33eb221ac2eb None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:51.506337 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:51.544340 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:51.569853 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ae86e2-10b8-4bdd-be34-3fd82a809249 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:51.581268 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1911378-3c97-47b0-a839-33eb221ac2eb None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:51.604874 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4635ed4c-c417-4996-a616-8ec0adb0a6e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:51.605531 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4635ed4c-c417-4996-a616-8ec0adb0a6e8 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:51.605811 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4635ed4c-c417-4996-a616-8ec0adb0a6e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:51.606083 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4635ed4c-c417-4996-a616-8ec0adb0a6e8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:51.612742 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4635ed4c-c417-4996-a616-8ec0adb0a6e8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:51.614255 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4635ed4c-c417-4996-a616-8ec0adb0a6e8 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:51.617995 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277f59d3-c834-412f-a473-d31d315ef0f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:51.618670 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277f59d3-c834-412f-a473-d31d315ef0f8 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:51.619017 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277f59d3-c834-412f-a473-d31d315ef0f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:51.619354 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277f59d3-c834-412f-a473-d31d315ef0f8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:51.626923 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-4635ed4c-c417-4996-a616-8ec0adb0a6e8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 08:23:51.626923 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:51.626923 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:51.626923 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:51.626923 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:51.626923 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:51.626923 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:51.626923 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:51.626923 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:51.626923 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:51.626923 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:51.626923 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:51.626923 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 08:23:51.626923 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 08:23:51.626923 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 08:23:51.628053 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 08:23:51.628053 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:51.628053 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:51.628053 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:51.628053 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:51.628053 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:51.628053 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:51.628053 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:51.628053 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:51.628053 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:51.628053 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:51.628053 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:51.628053 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:51.628053 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:51.628053 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:51.628053 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:51.628053 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:51.629470 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 08:23:51.629470 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 08:23:51.629470 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 08:23:51.629470 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 08:23:51.629470 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 08:23:51.629470 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 08:23:51.629470 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 08:23:51.629470 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:23:51.629470 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 327/662] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Aug 10 08:23:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:51.629470 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af3593a-11dc-4ff9-9da1-e858f1e96f83 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:51.630075 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af3593a-11dc-4ff9-9da1-e858f1e96f83 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:51.630075 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af3593a-11dc-4ff9-9da1-e858f1e96f83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:51.647951 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-4af3593a-11dc-4ff9-9da1-e858f1e96f83 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:23:51.649101 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-4af3593a-11dc-4ff9-9da1-e858f1e96f83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:23:51.696820 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277f59d3-c834-412f-a473-d31d315ef0f8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:51.698320 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277f59d3-c834-412f-a473-d31d315ef0f8 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:51.720987 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 328/663] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:23:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 08:23:51.720987 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af3593a-11dc-4ff9-9da1-e858f1e96f83 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:51.750741 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:51.782279 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:51.819378 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 336/664] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Aug 10 08:23:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 08:23:51.820514 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277f59d3-c834-412f-a473-d31d315ef0f8 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:51.824800 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af3593a-11dc-4ff9-9da1-e858f1e96f83 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:51.861622 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf318bc3-4b3a-4fdb-8357-8e35c157e46e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:51.862220 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf318bc3-4b3a-4fdb-8357-8e35c157e46e None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:51.862496 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf318bc3-4b3a-4fdb-8357-8e35c157e46e None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:51.863266 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf318bc3-4b3a-4fdb-8357-8e35c157e46e None admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:51.864446 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf318bc3-4b3a-4fdb-8357-8e35c157e46e None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:51.864750 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf318bc3-4b3a-4fdb-8357-8e35c157e46e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:51.866207 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf318bc3-4b3a-4fdb-8357-8e35c157e46e None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:51.885091 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 329/665] 10.222.0.57 () {56 vars in 1139 bytes} [Wed Aug 10 08:23:51 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:51.885885 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf318bc3-4b3a-4fdb-8357-8e35c157e46e None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:51.935784 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277f59d3-c834-412f-a473-d31d315ef0f8 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:51.971162 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:51.973522 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c09a318-d407-45a1-8711-b3865a24139d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:51.974133 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c09a318-d407-45a1-8711-b3865a24139d None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:51.974407 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c09a318-d407-45a1-8711-b3865a24139d None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:51.974674 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c09a318-d407-45a1-8711-b3865a24139d None admin] PATH_INFO: `/v3/projects` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:51.975457 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c09a318-d407-45a1-8711-b3865a24139d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:51.975750 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c09a318-d407-45a1-8711-b3865a24139d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:51.977198 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c09a318-d407-45a1-8711-b3865a24139d None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:51.989355 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf318bc3-4b3a-4fdb-8357-8e35c157e46e None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:51.995200 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 337/666] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Aug 10 08:23:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:51.995200 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c09a318-d407-45a1-8711-b3865a24139d None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:52.023522 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:52.036437 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c21b12-123e-487f-bcec-23036bc90aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:52.037173 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c21b12-123e-487f-bcec-23036bc90aa2 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:52.037527 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c21b12-123e-487f-bcec-23036bc90aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:52.037867 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c21b12-123e-487f-bcec-23036bc90aa2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:52.048820 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c21b12-123e-487f-bcec-23036bc90aa2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 08:23:52.054436 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-c0c21b12-123e-487f-bcec-23036bc90aa2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 08:23:52.054436 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:52.054436 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:52.054436 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:52.054436 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:52.054436 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:52.054436 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:52.054436 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:52.054436 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:52.054436 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:52.054436 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:52.054436 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:52.054436 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 08:23:52.054436 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 08:23:52.054436 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 08:23:52.054436 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 08:23:52.055607 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 08:23:52.055607 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 08:23:52.055607 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 08:23:52.055607 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 08:23:52.055607 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:52.055607 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:52.055607 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:52.055607 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:52.055607 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:52.055607 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:52.055607 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:52.055607 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:52.055607 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:52.055607 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:52.055607 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:52.055607 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:52.056765 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:52.056765 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:52.056765 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:52.056765 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:52.056765 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 08:23:52.056765 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 08:23:52.056765 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 08:23:52.056765 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 08:23:52.056765 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 08:23:52.056765 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 08:23:52.056765 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 08:23:52.056765 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:23:52.060291 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 330/667] 10.222.0.57 () {56 vars in 1102 bytes} [Wed Aug 10 08:23:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:52.061151 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c21b12-123e-487f-bcec-23036bc90aa2 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:52.105610 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c09a318-d407-45a1-8711-b3865a24139d None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:52.144930 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d243749-67b3-4187-85a0-3ddd26a5c22c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:52.145588 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d243749-67b3-4187-85a0-3ddd26a5c22c None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:52.145879 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d243749-67b3-4187-85a0-3ddd26a5c22c None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:52.146104 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d243749-67b3-4187-85a0-3ddd26a5c22c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:52.156643 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d243749-67b3-4187-85a0-3ddd26a5c22c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:52.158338 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d243749-67b3-4187-85a0-3ddd26a5c22c None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:52.169193 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c21b12-123e-487f-bcec-23036bc90aa2 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:52.173844 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-4d243749-67b3-4187-85a0-3ddd26a5c22c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:23:52.173844 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:52.173844 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:52.173844 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:52.173844 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:52.173844 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:52.173844 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:52.173844 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:52.173844 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:52.173844 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:52.173844 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:52.173844 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:52.173844 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 08:23:52.173844 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 08:23:52.173844 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 08:23:52.173844 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 08:23:52.174794 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:52.174794 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:52.174794 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:52.174794 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:52.174794 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:52.174794 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:52.174794 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:52.174794 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:52.174794 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:52.174794 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:52.174794 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:52.174794 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:52.174794 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:52.174794 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:52.174794 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:52.174794 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:52.174794 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 08:23:52.176361 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 08:23:52.176361 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:23:52.176361 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:23:52.180168 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 338/668] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:23:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:52.181048 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d243749-67b3-4187-85a0-3ddd26a5c22c None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:52.206336 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4a0842-9941-4ac9-a6d5-b87829e2765a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:52.207088 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4a0842-9941-4ac9-a6d5-b87829e2765a None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:52.207283 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4a0842-9941-4ac9-a6d5-b87829e2765a None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:52.207605 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4a0842-9941-4ac9-a6d5-b87829e2765a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:52.225367 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:52.262365 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:52.282332 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4a0842-9941-4ac9-a6d5-b87829e2765a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:52.283772 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4a0842-9941-4ac9-a6d5-b87829e2765a None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:52.288164 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d243749-67b3-4187-85a0-3ddd26a5c22c None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:52.328814 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2fad25-4637-4043-a146-1bbbcf19e084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:52.329927 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2fad25-4637-4043-a146-1bbbcf19e084 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:52.330358 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2fad25-4637-4043-a146-1bbbcf19e084 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:52.331188 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2fad25-4637-4043-a146-1bbbcf19e084 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:52.333086 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2fad25-4637-4043-a146-1bbbcf19e084 None admin] RBAC: Adding query filter params (role.id=73fdd60e2f804ee69b76fa9145a472ff, scope.project.id=b49e381419e940278bcb6111899589b5, user.id=f1ba74e28ced4cbba857f54222dfbe41) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:52.333795 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2fad25-4637-4043-a146-1bbbcf19e084 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:52.335767 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2fad25-4637-4043-a146-1bbbcf19e084 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:52.344111 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 339/669] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Aug 10 08:23:52 2022] GET /identity/v3/role_assignments?user.id=f1ba74e28ced4cbba857f54222dfbe41&scope.project.id=b49e381419e940278bcb6111899589b5&role.id=73fdd60e2f804ee69b76fa9145a472ff => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:52.345289 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2fad25-4637-4043-a146-1bbbcf19e084 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:52.395463 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 331/670] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Aug 10 08:23:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 08:23:52.396480 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4a0842-9941-4ac9-a6d5-b87829e2765a None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:52.454046 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:52.514782 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2fad25-4637-4043-a146-1bbbcf19e084 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:52.515303 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4a0842-9941-4ac9-a6d5-b87829e2765a None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:52.557231 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa01f9e-ff4a-4d05-8490-942d62057af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:52.558140 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa01f9e-ff4a-4d05-8490-942d62057af0 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:52.558571 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa01f9e-ff4a-4d05-8490-942d62057af0 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:52.558992 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa01f9e-ff4a-4d05-8490-942d62057af0 None admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:52.559834 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa01f9e-ff4a-4d05-8490-942d62057af0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:52.561067 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa01f9e-ff4a-4d05-8490-942d62057af0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:52.561193 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd2708c-e428-44ec-93dc-3d0fbee5d11b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:52.561412 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd2708c-e428-44ec-93dc-3d0fbee5d11b None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:52.561692 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd2708c-e428-44ec-93dc-3d0fbee5d11b None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:52.561963 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd2708c-e428-44ec-93dc-3d0fbee5d11b None admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:52.562399 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa01f9e-ff4a-4d05-8490-942d62057af0 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:52.562620 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd2708c-e428-44ec-93dc-3d0fbee5d11b None admin] RBAC: Adding query filter params (name=service) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:52.562921 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd2708c-e428-44ec-93dc-3d0fbee5d11b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:52.564568 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd2708c-e428-44ec-93dc-3d0fbee5d11b None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:52.575799 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 332/671] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Aug 10 08:23:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:52.577906 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 340/672] 10.222.0.57 () {56 vars in 1103 bytes} [Wed Aug 10 08:23:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:52.578978 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa01f9e-ff4a-4d05-8490-942d62057af0 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:52.581968 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd2708c-e428-44ec-93dc-3d0fbee5d11b None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:52.587226 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:52.690380 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa01f9e-ff4a-4d05-8490-942d62057af0 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:52.698111 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:52.705162 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd2708c-e428-44ec-93dc-3d0fbee5d11b None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:52.726588 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a07920-1205-4784-b614-d37e72c628f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:52.727288 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a07920-1205-4784-b614-d37e72c628f1 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:52.727534 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a07920-1205-4784-b614-d37e72c628f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:52.727805 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a07920-1205-4784-b614-d37e72c628f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:52.740381 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a07920-1205-4784-b614-d37e72c628f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:52.741917 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a07920-1205-4784-b614-d37e72c628f1 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:52.744791 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66675495-314e-4726-9811-e67b663cedc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:52.745938 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66675495-314e-4726-9811-e67b663cedc0 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:52.746442 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66675495-314e-4726-9811-e67b663cedc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:52.746911 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66675495-314e-4726-9811-e67b663cedc0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:52.754886 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66675495-314e-4726-9811-e67b663cedc0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:52.756611 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66675495-314e-4726-9811-e67b663cedc0 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:52.757622 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-42a07920-1205-4784-b614-d37e72c628f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:23:52.757622 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:52.757622 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:52.757622 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:52.757622 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:52.757622 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:52.757622 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:52.757622 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:52.757622 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:52.757622 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:52.757622 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:52.757622 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:52.757622 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 08:23:52.757622 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 08:23:52.757622 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 08:23:52.757622 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 08:23:52.758639 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:52.758639 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:52.758639 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:52.758639 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:52.758639 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:52.758639 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:52.758639 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:52.758639 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:52.758639 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:52.758639 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:52.758639 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:52.758639 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:52.758639 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:52.758639 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:52.758639 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:52.758639 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:52.758639 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 08:23:52.759558 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 08:23:52.759558 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:23:52.759558 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:23:52.763357 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 333/673] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:23:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:52.764206 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a07920-1205-4784-b614-d37e72c628f1 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:52.772568 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-66675495-314e-4726-9811-e67b663cedc0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 08:23:52.772568 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:52.772568 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:52.772568 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:52.772568 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:52.772568 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:52.772568 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:52.772568 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:52.772568 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:52.772568 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:52.772568 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:52.772568 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:52.772568 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 08:23:52.772568 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 08:23:52.772568 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 08:23:52.773491 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 08:23:52.773491 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:52.773491 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:52.773491 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:52.773491 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:52.773491 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:52.773491 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:52.773491 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:52.773491 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:52.773491 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:52.773491 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:52.773491 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:52.773491 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:52.773491 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:52.773491 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:52.773491 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:52.773491 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:52.774419 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 08:23:52.774419 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 08:23:52.774419 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 08:23:52.774419 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 08:23:52.774419 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 08:23:52.774419 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 08:23:52.774419 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 08:23:52.774419 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:23:52.781307 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 341/674] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Aug 10 08:23:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:52.782522 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66675495-314e-4726-9811-e67b663cedc0 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:52.849966 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:52.872049 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a07920-1205-4784-b614-d37e72c628f1 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:52.893485 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66675495-314e-4726-9811-e67b663cedc0 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:52.910645 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72d1adc-4022-4392-bafe-bcbe01b1731f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:52.911311 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72d1adc-4022-4392-bafe-bcbe01b1731f None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:52.911547 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72d1adc-4022-4392-bafe-bcbe01b1731f None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:52.911817 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72d1adc-4022-4392-bafe-bcbe01b1731f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:52.923382 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72d1adc-4022-4392-bafe-bcbe01b1731f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:52.925007 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72d1adc-4022-4392-bafe-bcbe01b1731f None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:52.932909 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190f5f35-2a67-44ab-965e-97d87315a35b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:52.933943 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190f5f35-2a67-44ab-965e-97d87315a35b None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:52.934444 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190f5f35-2a67-44ab-965e-97d87315a35b None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:52.934922 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190f5f35-2a67-44ab-965e-97d87315a35b None admin] PATH_INFO: `/v3/domains` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:52.935762 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190f5f35-2a67-44ab-965e-97d87315a35b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:52.936265 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190f5f35-2a67-44ab-965e-97d87315a35b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:52.937849 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190f5f35-2a67-44ab-965e-97d87315a35b None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:52.940191 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-d72d1adc-4022-4392-bafe-bcbe01b1731f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:23:52.940191 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:52.940191 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:52.940191 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:52.940191 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:52.940191 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:52.940191 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:52.940191 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:52.940191 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:52.940191 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:52.940191 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:52.940191 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:52.940191 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 08:23:52.940191 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 08:23:52.940191 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 08:23:52.940191 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 08:23:52.941675 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:52.941675 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:52.941675 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:52.941675 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:52.941675 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:52.941675 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:52.941675 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:52.941675 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:52.941675 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:52.941675 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:52.941675 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:52.941675 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:52.941675 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:52.941675 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:52.941675 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:52.941675 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:52.941675 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 08:23:52.942684 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 08:23:52.942684 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:23:52.942684 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:23:52.946828 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 334/675] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:23:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:52.947706 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72d1adc-4022-4392-bafe-bcbe01b1731f None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:52.954592 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 342/676] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Aug 10 08:23:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:52.955715 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190f5f35-2a67-44ab-965e-97d87315a35b None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:52.961868 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:53.057176 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72d1adc-4022-4392-bafe-bcbe01b1731f None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:53.063994 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190f5f35-2a67-44ab-965e-97d87315a35b None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:53.090419 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:53.094179 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26371241-0d34-466a-a866-6bb2d4d026b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:53.094785 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26371241-0d34-466a-a866-6bb2d4d026b5 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:53.095068 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26371241-0d34-466a-a866-6bb2d4d026b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:53.095340 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26371241-0d34-466a-a866-6bb2d4d026b5 None admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:53.096038 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26371241-0d34-466a-a866-6bb2d4d026b5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:53.096350 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26371241-0d34-466a-a866-6bb2d4d026b5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:53.097997 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26371241-0d34-466a-a866-6bb2d4d026b5 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:53.101707 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae942e67-1204-4c61-8906-b14a124fae11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:53.102313 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae942e67-1204-4c61-8906-b14a124fae11 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:53.103550 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae942e67-1204-4c61-8906-b14a124fae11 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:53.103550 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae942e67-1204-4c61-8906-b14a124fae11 None admin] PATH_INFO: `/v3/domains` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:53.103717 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae942e67-1204-4c61-8906-b14a124fae11 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:53.103875 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae942e67-1204-4c61-8906-b14a124fae11 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:53.105247 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae942e67-1204-4c61-8906-b14a124fae11 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:53.116275 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 335/677] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Aug 10 08:23:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:53.116275 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26371241-0d34-466a-a866-6bb2d4d026b5 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:53.124644 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 343/678] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Aug 10 08:23:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:53.129103 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522ee68d-f825-4b82-b435-71a18a3e9fc1 None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:53.129103 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522ee68d-f825-4b82-b435-71a18a3e9fc1 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:53.129103 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522ee68d-f825-4b82-b435-71a18a3e9fc1 None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:53.133655 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 344/679] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:23:53 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:23:53.134688 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522ee68d-f825-4b82-b435-71a18a3e9fc1 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:53.199862 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:53.232876 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26371241-0d34-466a-a866-6bb2d4d026b5 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:53.249570 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522ee68d-f825-4b82-b435-71a18a3e9fc1 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:53.269788 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e54dc0-e1b8-4b6f-ac15-93aef53cfe7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:53.270414 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e54dc0-e1b8-4b6f-ac15-93aef53cfe7e None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:53.270688 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e54dc0-e1b8-4b6f-ac15-93aef53cfe7e None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:53.270951 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e54dc0-e1b8-4b6f-ac15-93aef53cfe7e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:53.277589 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e54dc0-e1b8-4b6f-ac15-93aef53cfe7e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:53.279016 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e54dc0-e1b8-4b6f-ac15-93aef53cfe7e None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:53.285590 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008fee61-47fc-4ffd-964f-4c27043c22f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:53.286312 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008fee61-47fc-4ffd-964f-4c27043c22f7 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:53.286593 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008fee61-47fc-4ffd-964f-4c27043c22f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:53.286863 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008fee61-47fc-4ffd-964f-4c27043c22f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:53.287577 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008fee61-47fc-4ffd-964f-4c27043c22f7 None admin] RBAC: Adding query filter params (role.id=73fdd60e2f804ee69b76fa9145a472ff, scope.project.id=b49e381419e940278bcb6111899589b5, user.id=a943008193914f76a663d489bd73e873) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:53.287881 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008fee61-47fc-4ffd-964f-4c27043c22f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:53.289472 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008fee61-47fc-4ffd-964f-4c27043c22f7 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:53.290109 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-e4e54dc0-e1b8-4b6f-ac15-93aef53cfe7e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 08:23:53.290109 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:53.290109 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:53.290109 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:53.290109 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:53.290109 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:53.290109 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:53.290109 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:53.290109 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:53.290109 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:53.290109 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:53.290109 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:53.290109 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 08:23:53.290109 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 08:23:53.290109 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 08:23:53.291020 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 08:23:53.291020 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:53.291020 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:53.291020 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:53.291020 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:53.291020 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:53.291020 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:53.291020 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:53.291020 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:53.291020 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:53.291020 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:53.291020 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:53.291020 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:53.291020 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:53.291020 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:53.291020 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:53.291020 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:53.291925 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 08:23:53.291925 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 08:23:53.291925 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 08:23:53.291925 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 08:23:53.291925 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 08:23:53.291925 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 08:23:53.291925 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 08:23:53.291925 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:23:53.291925 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 336/680] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Aug 10 08:23:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:53.291925 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e54dc0-e1b8-4b6f-ac15-93aef53cfe7e None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:53.300780 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 345/681] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Aug 10 08:23:53 2022] GET /identity/v3/role_assignments?user.id=a943008193914f76a663d489bd73e873&scope.project.id=b49e381419e940278bcb6111899589b5&role.id=73fdd60e2f804ee69b76fa9145a472ff => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:53.306757 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbdd61b-9159-4323-955a-f30c5b5d871d None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:53.307710 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbdd61b-9159-4323-955a-f30c5b5d871d None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:53.308269 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbdd61b-9159-4323-955a-f30c5b5d871d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:53.335298 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-ddbdd61b-9159-4323-955a-f30c5b5d871d None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:23:53.336706 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-ddbdd61b-9159-4323-955a-f30c5b5d871d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:23:53.394884 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:53.428945 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 346/682] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:23:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 08:23:53.428945 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbdd61b-9159-4323-955a-f30c5b5d871d None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:53.448427 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e54dc0-e1b8-4b6f-ac15-93aef53cfe7e None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:53.489126 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:53.496143 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9043944-72cc-46a9-9d75-3e3ffdf42c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:53.496780 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9043944-72cc-46a9-9d75-3e3ffdf42c3a None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:53.497113 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9043944-72cc-46a9-9d75-3e3ffdf42c3a None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:53.497386 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9043944-72cc-46a9-9d75-3e3ffdf42c3a None admin] PATH_INFO: `/v3/users/swift` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:53.506538 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9043944-72cc-46a9-9d75-3e3ffdf42c3a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:53.507990 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9043944-72cc-46a9-9d75-3e3ffdf42c3a None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:53.523413 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-b9043944-72cc-46a9-9d75-3e3ffdf42c3a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 10 08:23:53.523413 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:53.523413 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:53.523413 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:53.523413 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:53.523413 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:53.523413 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:53.523413 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:53.523413 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:53.523413 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:53.523413 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:53.523413 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:53.523413 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 08:23:53.523413 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 08:23:53.523413 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 08:23:53.523413 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 08:23:53.524504 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:53.524504 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:53.524504 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 08:23:53.524504 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 08:23:53.524504 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 08:23:53.524504 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 08:23:53.524504 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:53.524504 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:53.524504 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:53.524504 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:53.524504 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:53.524504 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:53.524504 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:53.524504 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:53.524504 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:53.524504 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:53.524504 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:53.524504 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:53.525515 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:53.525515 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:53.525515 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 08:23:53.525515 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 08:23:53.525515 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 08:23:53.525515 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 08:23:53.525515 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 08:23:53.525515 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 08:23:53.525515 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 10 08:23:53.525515 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:23:53.525515 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 337/683] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Aug 10 08:23:53 2022] GET /identity/v3/users/swift => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:53.526967 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9043944-72cc-46a9-9d75-3e3ffdf42c3a None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:53.587982 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbdd61b-9159-4323-955a-f30c5b5d871d None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:53.634466 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2362e60c-9f6b-4f81-bf2d-b1b858d18b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:53.635081 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2362e60c-9f6b-4f81-bf2d-b1b858d18b11 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:53.635370 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2362e60c-9f6b-4f81-bf2d-b1b858d18b11 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:53.635622 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2362e60c-9f6b-4f81-bf2d-b1b858d18b11 None admin] PATH_INFO: `/v3/projects` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:53.636315 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2362e60c-9f6b-4f81-bf2d-b1b858d18b11 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:53.636615 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2362e60c-9f6b-4f81-bf2d-b1b858d18b11 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:53.638378 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2362e60c-9f6b-4f81-bf2d-b1b858d18b11 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:53.652844 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9043944-72cc-46a9-9d75-3e3ffdf42c3a None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:53.653768 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 347/684] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Aug 10 08:23:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:53.654613 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2362e60c-9f6b-4f81-bf2d-b1b858d18b11 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:53.717383 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:53.726492 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a5a610-bab1-499a-b8a9-f3cdda007529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:53.729524 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a5a610-bab1-499a-b8a9-f3cdda007529 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:53.730448 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a5a610-bab1-499a-b8a9-f3cdda007529 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:53.730922 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a5a610-bab1-499a-b8a9-f3cdda007529 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:53.748159 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a5a610-bab1-499a-b8a9-f3cdda007529 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:53.750242 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a5a610-bab1-499a-b8a9-f3cdda007529 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:53.771920 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-a3a5a610-bab1-499a-b8a9-f3cdda007529 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:23:53.771920 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:53.771920 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:53.771920 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:53.771920 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:53.771920 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:53.771920 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:53.771920 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:53.771920 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:53.771920 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:53.771920 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:53.771920 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:53.771920 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 08:23:53.771920 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 08:23:53.771920 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 08:23:53.771920 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 08:23:53.773381 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:53.773381 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:53.773381 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:53.773381 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:53.773381 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:53.773381 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:53.773381 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:53.773381 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:53.773381 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:53.773381 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:53.773381 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:53.773381 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:53.773381 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:53.773381 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:53.773381 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:53.773381 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:53.773381 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 08:23:53.774301 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 08:23:53.774301 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:23:53.774301 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:23:53.775239 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2362e60c-9f6b-4f81-bf2d-b1b858d18b11 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:53.793058 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 338/685] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:23:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:53.794261 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a5a610-bab1-499a-b8a9-f3cdda007529 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:53.814402 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582b1525-5148-4baf-ac2c-0c608e43e0d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:53.815010 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582b1525-5148-4baf-ac2c-0c608e43e0d6 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:53.815276 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582b1525-5148-4baf-ac2c-0c608e43e0d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:53.815543 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582b1525-5148-4baf-ac2c-0c608e43e0d6 None admin] PATH_INFO: `/v3/users` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:53.816645 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582b1525-5148-4baf-ac2c-0c608e43e0d6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:53.816977 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582b1525-5148-4baf-ac2c-0c608e43e0d6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:53.818406 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582b1525-5148-4baf-ac2c-0c608e43e0d6 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:53.841134 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 348/686] 10.222.0.57 () {56 vars in 1135 bytes} [Wed Aug 10 08:23:53 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:53.841134 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582b1525-5148-4baf-ac2c-0c608e43e0d6 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:53.875716 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:54.000829 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a5a610-bab1-499a-b8a9-f3cdda007529 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:54.021949 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:54.032258 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582b1525-5148-4baf-ac2c-0c608e43e0d6 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:54.057313 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2cd2c8-4be9-4ff5-b501-84a8e5e5e9af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:54.058691 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2cd2c8-4be9-4ff5-b501-84a8e5e5e9af None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:54.059504 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2cd2c8-4be9-4ff5-b501-84a8e5e5e9af None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:54.060304 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2cd2c8-4be9-4ff5-b501-84a8e5e5e9af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:54.063778 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2cd2c8-4be9-4ff5-b501-84a8e5e5e9af None admin] RBAC: Adding query filter params (role.id=73fdd60e2f804ee69b76fa9145a472ff, scope.project.id=b49e381419e940278bcb6111899589b5, user.id=d4863ecff21e408c9deca645b88d3727) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:54.064437 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2cd2c8-4be9-4ff5-b501-84a8e5e5e9af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:54.068092 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2cd2c8-4be9-4ff5-b501-84a8e5e5e9af None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:54.081930 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 339/687] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Aug 10 08:23:53 2022] GET /identity/v3/role_assignments?user.id=d4863ecff21e408c9deca645b88d3727&scope.project.id=b49e381419e940278bcb6111899589b5&role.id=73fdd60e2f804ee69b76fa9145a472ff => generated 608 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:54.083494 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2cd2c8-4be9-4ff5-b501-84a8e5e5e9af None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:54.107200 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15420514-0ed7-4abb-bf13-df18cacdb60d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:54.107754 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15420514-0ed7-4abb-bf13-df18cacdb60d None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:54.108029 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15420514-0ed7-4abb-bf13-df18cacdb60d None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:54.108295 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15420514-0ed7-4abb-bf13-df18cacdb60d None admin] PATH_INFO: `/v3/domains` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:54.109533 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15420514-0ed7-4abb-bf13-df18cacdb60d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:54.110124 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15420514-0ed7-4abb-bf13-df18cacdb60d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:54.112488 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15420514-0ed7-4abb-bf13-df18cacdb60d None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:54.135356 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 349/688] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Aug 10 08:23:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:54.141103 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15420514-0ed7-4abb-bf13-df18cacdb60d None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:54.296854 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2cd2c8-4be9-4ff5-b501-84a8e5e5e9af None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:54.306712 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:54.336999 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790ca54f-50b1-450b-ab73-c85abe0a9ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:54.338056 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790ca54f-50b1-450b-ab73-c85abe0a9ddf None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:54.338408 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790ca54f-50b1-450b-ab73-c85abe0a9ddf None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:54.338680 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790ca54f-50b1-450b-ab73-c85abe0a9ddf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:54.395832 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15420514-0ed7-4abb-bf13-df18cacdb60d None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:54.418379 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790ca54f-50b1-450b-ab73-c85abe0a9ddf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:54.419952 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790ca54f-50b1-450b-ab73-c85abe0a9ddf None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:54.432407 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5528e833-12a3-4245-81c3-d098441ccadd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:54.433221 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5528e833-12a3-4245-81c3-d098441ccadd None admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:54.433500 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5528e833-12a3-4245-81c3-d098441ccadd None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:54.433771 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5528e833-12a3-4245-81c3-d098441ccadd None admin] PATH_INFO: `/v3/users` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:54.434416 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5528e833-12a3-4245-81c3-d098441ccadd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:54.436004 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5528e833-12a3-4245-81c3-d098441ccadd None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:54.530195 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 340/689] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Aug 10 08:23:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 08:23:54.537946 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790ca54f-50b1-450b-ab73-c85abe0a9ddf None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:54.546658 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:54.565441 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 350/690] 10.222.0.57 () {60 vars in 1132 bytes} [Wed Aug 10 08:23:54 2022] POST /identity/v3/users => generated 254 bytes in 424 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 08:23:54.569067 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1353a66-1554-42de-8f01-4f1a6e9ea6e0 None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:54.569362 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1353a66-1554-42de-8f01-4f1a6e9ea6e0 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:54.569642 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1353a66-1554-42de-8f01-4f1a6e9ea6e0 None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:54.572609 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 351/691] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:23:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:23:54.577256 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bb5cc1-aa39-4f60-bc21-14468004c13b None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:54.577624 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bb5cc1-aa39-4f60-bc21-14468004c13b None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:54.577972 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bb5cc1-aa39-4f60-bc21-14468004c13b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:54.625618 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-72bb5cc1-aa39-4f60-bc21-14468004c13b None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:23:54.626965 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-72bb5cc1-aa39-4f60-bc21-14468004c13b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:23:54.636259 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:54.660075 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790ca54f-50b1-450b-ab73-c85abe0a9ddf None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:54.735841 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1a4c97-6c25-4afd-b57e-3032128ed727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:54.737477 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1a4c97-6c25-4afd-b57e-3032128ed727 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:54.738575 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1a4c97-6c25-4afd-b57e-3032128ed727 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:54.739084 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1a4c97-6c25-4afd-b57e-3032128ed727 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:54.759889 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1a4c97-6c25-4afd-b57e-3032128ed727 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:54.762598 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1a4c97-6c25-4afd-b57e-3032128ed727 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:54.771759 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 352/692] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:23:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 08:23:54.785444 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-ae1a4c97-6c25-4afd-b57e-3032128ed727 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:23:54.785444 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:54.785444 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:54.785444 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:54.785444 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:54.785444 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:54.785444 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:54.785444 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:54.785444 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:54.785444 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:54.785444 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:54.785444 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:54.785444 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 08:23:54.785444 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 08:23:54.785444 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 08:23:54.785444 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 08:23:54.786482 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:54.786482 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:54.786482 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:54.786482 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:54.786482 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:54.786482 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:54.786482 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:54.786482 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:54.786482 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:54.786482 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:54.786482 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:54.786482 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:54.786482 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:54.786482 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:54.786482 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:54.786482 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:54.786482 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 08:23:54.787426 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 08:23:54.787426 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:23:54.787426 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:23:54.789575 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 341/693] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:23:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:54.795397 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bb5cc1-aa39-4f60-bc21-14468004c13b None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:54.799470 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1a4c97-6c25-4afd-b57e-3032128ed727 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:54.864653 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:54.910749 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bb5cc1-aa39-4f60-bc21-14468004c13b None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:54.917719 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1a4c97-6c25-4afd-b57e-3032128ed727 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:54.950707 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4875898-35be-449f-882c-ddcb52a35c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:54.951856 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4875898-35be-449f-882c-ddcb52a35c51 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:54.952398 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4875898-35be-449f-882c-ddcb52a35c51 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:54.952653 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd56312f-c18e-473d-8ac4-769edea3903b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:54.953144 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4875898-35be-449f-882c-ddcb52a35c51 None admin] PATH_INFO: `/v3/domains` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:54.953342 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd56312f-c18e-473d-8ac4-769edea3903b None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:54.953576 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd56312f-c18e-473d-8ac4-769edea3903b None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:54.953843 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd56312f-c18e-473d-8ac4-769edea3903b None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:54.954485 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd56312f-c18e-473d-8ac4-769edea3903b None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:54.954923 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4875898-35be-449f-882c-ddcb52a35c51 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:54.955430 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4875898-35be-449f-882c-ddcb52a35c51 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:54.955832 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd56312f-c18e-473d-8ac4-769edea3903b None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:54.957766 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4875898-35be-449f-882c-ddcb52a35c51 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:54.967882 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-fd56312f-c18e-473d-8ac4-769edea3903b None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 08:23:54.967882 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:54.967882 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 08:23:54.967882 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 08:23:54.967882 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 08:23:54.967882 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 08:23:54.967882 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 08:23:54.967882 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 08:23:54.967882 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 08:23:54.967882 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:23:54.967882 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 08:23:54.967882 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:23:54.967882 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:54.967882 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:54.967882 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:54.967882 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:54.967882 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:54.969016 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:54.969016 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:54.969016 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:54.969016 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:54.969016 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:54.969016 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:54.969016 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 08:23:54.969016 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 08:23:54.969016 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 08:23:54.969016 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 08:23:54.969016 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:54.969016 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:54.969016 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:54.969016 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:54.969016 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:54.969016 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:54.969889 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:54.969889 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:54.969889 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:54.969889 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:54.969889 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:54.969889 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:54.969889 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:54.969889 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:54.969889 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:54.969889 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:54.969889 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 08:23:54.969889 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 08:23:54.969889 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 08:23:54.969889 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:23:54.969889 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 342/694] 10.222.0.57 () {56 vars in 1129 bytes} [Wed Aug 10 08:23:54 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 08:23:54.973883 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 353/695] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Aug 10 08:23:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:54.979423 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd56312f-c18e-473d-8ac4-769edea3903b None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:54.980026 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4875898-35be-449f-882c-ddcb52a35c51 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:55.037299 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:55.085557 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd56312f-c18e-473d-8ac4-769edea3903b None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:55.109918 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:55.121064 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4875898-35be-449f-882c-ddcb52a35c51 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:55.132186 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97808a4-eb47-4f68-a0e2-231dd9326e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:55.133285 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97808a4-eb47-4f68-a0e2-231dd9326e82 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:55.133709 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97808a4-eb47-4f68-a0e2-231dd9326e82 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:55.134128 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97808a4-eb47-4f68-a0e2-231dd9326e82 None admin] PATH_INFO: `/v3/services` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:55.134902 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97808a4-eb47-4f68-a0e2-231dd9326e82 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:55.135344 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97808a4-eb47-4f68-a0e2-231dd9326e82 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:55.136849 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97808a4-eb47-4f68-a0e2-231dd9326e82 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:55.145382 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 343/696] 10.222.0.57 () {56 vars in 1124 bytes} [Wed Aug 10 08:23:54 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:55.152209 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97808a4-eb47-4f68-a0e2-231dd9326e82 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:55.195103 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f236bab6-0023-4d2f-ac26-bd1e4642c0ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:55.195966 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f236bab6-0023-4d2f-ac26-bd1e4642c0ff None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:55.196443 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f236bab6-0023-4d2f-ac26-bd1e4642c0ff None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:55.196852 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f236bab6-0023-4d2f-ac26-bd1e4642c0ff None admin] PATH_INFO: `/v3/projects/service` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:55.212515 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f236bab6-0023-4d2f-ac26-bd1e4642c0ff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:55.215069 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f236bab6-0023-4d2f-ac26-bd1e4642c0ff None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:55.238757 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-f236bab6-0023-4d2f-ac26-bd1e4642c0ff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 08:23:55.238757 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:55.238757 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:55.238757 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:55.238757 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:55.238757 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:55.238757 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:55.238757 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:55.238757 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:55.238757 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:55.238757 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:55.238757 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:55.238757 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 08:23:55.238757 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 08:23:55.238757 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 08:23:55.239720 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 08:23:55.239720 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:55.239720 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:55.239720 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:55.239720 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:55.239720 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:55.239720 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:55.239720 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:55.239720 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:55.239720 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:55.239720 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:55.239720 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:55.239720 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:55.239720 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:55.239720 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:55.239720 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:55.239720 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:55.240610 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 08:23:55.240610 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 08:23:55.240610 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 08:23:55.240610 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 08:23:55.240610 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 08:23:55.240610 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 08:23:55.240610 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 08:23:55.240610 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:23:55.240610 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 354/697] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Aug 10 08:23:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:55.246221 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f236bab6-0023-4d2f-ac26-bd1e4642c0ff None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:55.322739 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97808a4-eb47-4f68-a0e2-231dd9326e82 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:55.329311 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:55.361027 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f236bab6-0023-4d2f-ac26-bd1e4642c0ff None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:55.393383 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e7233f-be68-4c76-98d9-5391ad7021a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:55.394888 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e7233f-be68-4c76-98d9-5391ad7021a8 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:55.399969 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e7233f-be68-4c76-98d9-5391ad7021a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:55.399969 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e7233f-be68-4c76-98d9-5391ad7021a8 None admin] PATH_INFO: `/v3/services` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:55.399969 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e7233f-be68-4c76-98d9-5391ad7021a8 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:55.399969 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e7233f-be68-4c76-98d9-5391ad7021a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:55.400564 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e7233f-be68-4c76-98d9-5391ad7021a8 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:55.412161 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 344/698] 10.222.0.57 () {56 vars in 1124 bytes} [Wed Aug 10 08:23:55 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:55.425511 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66891c1f-3462-4319-ba66-b3bc5bd394f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:55.426139 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66891c1f-3462-4319-ba66-b3bc5bd394f1 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:55.426425 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66891c1f-3462-4319-ba66-b3bc5bd394f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:55.426701 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66891c1f-3462-4319-ba66-b3bc5bd394f1 None admin] PATH_INFO: `/v3/projects` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:55.427463 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66891c1f-3462-4319-ba66-b3bc5bd394f1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:55.427777 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66891c1f-3462-4319-ba66-b3bc5bd394f1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:55.429234 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66891c1f-3462-4319-ba66-b3bc5bd394f1 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:55.453350 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 355/699] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Aug 10 08:23:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:23:55.462112 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e7233f-be68-4c76-98d9-5391ad7021a8 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:55.519288 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:55.586483 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e7233f-be68-4c76-98d9-5391ad7021a8 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:55.638465 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5f4de2-c015-4e2a-9513-47ae50a06b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:55.639085 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5f4de2-c015-4e2a-9513-47ae50a06b3c None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:55.639361 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5f4de2-c015-4e2a-9513-47ae50a06b3c None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:55.639632 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5f4de2-c015-4e2a-9513-47ae50a06b3c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:55.640369 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5f4de2-c015-4e2a-9513-47ae50a06b3c None admin] RBAC: Adding query filter params (role.id=73fdd60e2f804ee69b76fa9145a472ff, scope.project.id=b49e381419e940278bcb6111899589b5, user.id=b4b26ba3a9e04c38b52db2a46198a53c) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:55.640675 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5f4de2-c015-4e2a-9513-47ae50a06b3c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:55.642144 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5f4de2-c015-4e2a-9513-47ae50a06b3c None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:55.648883 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 345/700] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Aug 10 08:23:55 2022] GET /identity/v3/role_assignments?user.id=b4b26ba3a9e04c38b52db2a46198a53c&scope.project.id=b49e381419e940278bcb6111899589b5&role.id=73fdd60e2f804ee69b76fa9145a472ff => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:55.755648 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc70ee51-fa93-4812-b0ce-c425bf4d1900 None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:55.756165 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc70ee51-fa93-4812-b0ce-c425bf4d1900 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:55.756165 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc70ee51-fa93-4812-b0ce-c425bf4d1900 None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:55.757788 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 356/701] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:23:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:23:55.763450 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f531a64-a16a-40f6-9ee7-7df4fd39c351 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:55.763932 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f531a64-a16a-40f6-9ee7-7df4fd39c351 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:55.764007 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f531a64-a16a-40f6-9ee7-7df4fd39c351 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:55.786814 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-4f531a64-a16a-40f6-9ee7-7df4fd39c351 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:23:55.788020 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-4f531a64-a16a-40f6-9ee7-7df4fd39c351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:23:55.873816 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 346/702] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:23:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 08:23:55.900823 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc70ee51-fa93-4812-b0ce-c425bf4d1900 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:56.012091 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc70ee51-fa93-4812-b0ce-c425bf4d1900 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:56.038538 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:56.051877 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6137f633-8544-49b3-9a4d-4ebce10b4d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:56.052690 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6137f633-8544-49b3-9a4d-4ebce10b4d12 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:56.052791 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6137f633-8544-49b3-9a4d-4ebce10b4d12 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:56.053049 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6137f633-8544-49b3-9a4d-4ebce10b4d12 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:56.063526 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6137f633-8544-49b3-9a4d-4ebce10b4d12 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 08:23:56.068851 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-6137f633-8544-49b3-9a4d-4ebce10b4d12 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 08:23:56.068851 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:56.068851 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:56.068851 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:56.068851 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:56.068851 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:56.068851 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:56.068851 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:56.068851 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:56.068851 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:56.068851 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:56.068851 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:56.068851 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 08:23:56.068851 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 08:23:56.068851 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 08:23:56.068851 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 08:23:56.069862 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 08:23:56.069862 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 08:23:56.069862 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 08:23:56.069862 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 08:23:56.069862 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:56.069862 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:56.069862 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:56.069862 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:56.069862 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:56.069862 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:56.069862 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:56.069862 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:56.069862 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:56.069862 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:56.069862 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:56.069862 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:56.070723 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:56.070723 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:56.070723 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:56.070723 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:56.070723 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 08:23:56.070723 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 08:23:56.070723 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 08:23:56.070723 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 08:23:56.070723 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 08:23:56.070723 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 08:23:56.070723 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 08:23:56.070723 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:23:56.070723 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 357/703] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Aug 10 08:23:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:56.076308 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f531a64-a16a-40f6-9ee7-7df4fd39c351 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:56.138662 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:56.201140 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f531a64-a16a-40f6-9ee7-7df4fd39c351 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:56.248221 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b6e977-1fcb-46b8-999b-b84219e57885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:56.249031 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b6e977-1fcb-46b8-999b-b84219e57885 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:56.249411 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b6e977-1fcb-46b8-999b-b84219e57885 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:56.249780 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b6e977-1fcb-46b8-999b-b84219e57885 None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:56.250547 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b6e977-1fcb-46b8-999b-b84219e57885 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:56.250949 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b6e977-1fcb-46b8-999b-b84219e57885 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:56.252425 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b6e977-1fcb-46b8-999b-b84219e57885 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:56.261848 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 347/704] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Aug 10 08:23:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:56.271674 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6137f633-8544-49b3-9a4d-4ebce10b4d12 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:56.432637 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6137f633-8544-49b3-9a4d-4ebce10b4d12 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:56.486719 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7932dcd2-2fdc-45e6-8eaa-dbf962be7491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:56.487420 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7932dcd2-2fdc-45e6-8eaa-dbf962be7491 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:56.487823 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7932dcd2-2fdc-45e6-8eaa-dbf962be7491 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:56.488390 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7932dcd2-2fdc-45e6-8eaa-dbf962be7491 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:56.508819 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7932dcd2-2fdc-45e6-8eaa-dbf962be7491 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:56.510312 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7932dcd2-2fdc-45e6-8eaa-dbf962be7491 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:56.529791 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-7932dcd2-2fdc-45e6-8eaa-dbf962be7491 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:23:56.529791 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:56.529791 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:56.529791 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:56.529791 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:56.529791 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:56.529791 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:56.529791 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:56.529791 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:56.529791 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:56.529791 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:56.529791 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:56.529791 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 08:23:56.529791 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 08:23:56.529791 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 08:23:56.529791 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 08:23:56.530954 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:56.530954 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:56.530954 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:56.530954 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:56.530954 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:56.530954 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:56.530954 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:56.530954 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:56.530954 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:56.530954 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:56.530954 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:56.530954 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:56.530954 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:56.530954 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:56.530954 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:56.530954 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:56.530954 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 08:23:56.532103 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 08:23:56.532103 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:23:56.532103 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:23:56.532828 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 358/705] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:23:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:56.538498 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b6e977-1fcb-46b8-999b-b84219e57885 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:56.586343 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809c4fd0-bf96-4f34-97a2-cc5d988b961c None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:56.586628 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809c4fd0-bf96-4f34-97a2-cc5d988b961c None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:56.586901 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809c4fd0-bf96-4f34-97a2-cc5d988b961c None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:56.588525 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 359/706] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:23:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 08:23:56.597604 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71819f55-c0bf-454d-b434-388a05c868b9 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:56.597604 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71819f55-c0bf-454d-b434-388a05c868b9 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:56.597604 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71819f55-c0bf-454d-b434-388a05c868b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:56.630216 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-71819f55-c0bf-454d-b434-388a05c868b9 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:23:56.631678 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-71819f55-c0bf-454d-b434-388a05c868b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:23:56.673276 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:56.718277 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 360/707] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:23:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 08:23:56.736757 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b6e977-1fcb-46b8-999b-b84219e57885 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:56.741280 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71819f55-c0bf-454d-b434-388a05c868b9 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:56.748044 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:56.809883 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0471ff-c613-42e9-aae5-ba6a82a5807f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:56.813746 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0471ff-c613-42e9-aae5-ba6a82a5807f None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:56.814031 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0471ff-c613-42e9-aae5-ba6a82a5807f None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:56.814304 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0471ff-c613-42e9-aae5-ba6a82a5807f None admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:56.814952 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0471ff-c613-42e9-aae5-ba6a82a5807f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:56.815252 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0471ff-c613-42e9-aae5-ba6a82a5807f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:56.817293 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0471ff-c613-42e9-aae5-ba6a82a5807f None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:56.836272 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 348/708] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Aug 10 08:23:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:56.841911 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0471ff-c613-42e9-aae5-ba6a82a5807f None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:56.886931 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71819f55-c0bf-454d-b434-388a05c868b9 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:56.935066 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aff64bf-8c27-4fe5-80d4-75cca2d5ad0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:56.936235 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aff64bf-8c27-4fe5-80d4-75cca2d5ad0b None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:56.936715 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aff64bf-8c27-4fe5-80d4-75cca2d5ad0b None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:56.937231 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aff64bf-8c27-4fe5-80d4-75cca2d5ad0b None admin] PATH_INFO: `/v3/services/network` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:56.938091 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aff64bf-8c27-4fe5-80d4-75cca2d5ad0b None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:56.939676 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aff64bf-8c27-4fe5-80d4-75cca2d5ad0b None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:56.956049 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-4aff64bf-8c27-4fe5-80d4-75cca2d5ad0b None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 08:23:56.956049 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:56.956049 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 08:23:56.956049 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 08:23:56.956049 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 08:23:56.956049 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 08:23:56.956049 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 08:23:56.956049 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 08:23:56.956049 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 08:23:56.956049 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:23:56.956049 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 08:23:56.956049 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:23:56.956049 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:56.956049 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:56.956049 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:56.956049 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:56.956049 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:56.957180 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:56.957180 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:56.957180 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:56.957180 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:56.957180 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:56.957180 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:56.957180 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 08:23:56.957180 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 08:23:56.957180 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 08:23:56.957180 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 08:23:56.957180 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:56.957180 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:56.957180 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:56.957180 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:56.957180 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:56.957180 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:56.958045 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:56.958045 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:56.958045 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:56.958045 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:56.958045 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:56.958045 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:56.958045 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:56.958045 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:56.958045 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:56.958045 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:56.958045 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 08:23:56.958045 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 08:23:56.958045 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 08:23:56.958045 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:23:56.960952 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 361/709] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Aug 10 08:23:56 2022] GET /identity/v3/services/network => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:56.967420 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aff64bf-8c27-4fe5-80d4-75cca2d5ad0b None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:57.019485 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0471ff-c613-42e9-aae5-ba6a82a5807f None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:57.024349 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:57.102680 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed87485-3407-48b5-8554-428aed1549bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:57.103237 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed87485-3407-48b5-8554-428aed1549bd None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:57.103520 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed87485-3407-48b5-8554-428aed1549bd None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:57.103792 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed87485-3407-48b5-8554-428aed1549bd None admin] PATH_INFO: `/v3/users/nova` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:57.117256 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed87485-3407-48b5-8554-428aed1549bd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:57.118584 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed87485-3407-48b5-8554-428aed1549bd None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:57.131078 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aff64bf-8c27-4fe5-80d4-75cca2d5ad0b None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:57.142589 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-bed87485-3407-48b5-8554-428aed1549bd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 08:23:57.142589 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:57.142589 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:57.142589 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:57.142589 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:57.142589 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:57.142589 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:57.142589 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:57.142589 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:57.142589 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:57.142589 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:57.142589 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:57.142589 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 08:23:57.142589 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 08:23:57.142589 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 08:23:57.142589 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 08:23:57.143575 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:57.143575 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:57.143575 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 08:23:57.143575 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 08:23:57.143575 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 08:23:57.143575 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 08:23:57.143575 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:57.143575 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:57.143575 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:57.143575 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:57.143575 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:57.143575 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:57.143575 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:57.143575 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:57.143575 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:57.143575 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:57.143575 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:57.143575 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:57.144574 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:57.144574 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:57.144574 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 08:23:57.144574 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 08:23:57.144574 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 08:23:57.144574 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 08:23:57.144574 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 08:23:57.144574 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 08:23:57.144574 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 08:23:57.144574 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:23:57.148117 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 349/710] 10.222.0.57 () {56 vars in 1093 bytes} [Wed Aug 10 08:23:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:57.149545 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed87485-3407-48b5-8554-428aed1549bd None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:57.177950 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:57.181327 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719d9e18-06f6-438d-b959-014ae2b065b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:57.182214 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719d9e18-06f6-438d-b959-014ae2b065b8 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:57.182694 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719d9e18-06f6-438d-b959-014ae2b065b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:57.183163 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719d9e18-06f6-438d-b959-014ae2b065b8 None admin] PATH_INFO: `/v3/services` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:57.184022 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719d9e18-06f6-438d-b959-014ae2b065b8 None admin] RBAC: Adding query filter params (name=network) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:57.184526 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719d9e18-06f6-438d-b959-014ae2b065b8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:57.189186 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719d9e18-06f6-438d-b959-014ae2b065b8 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:57.194942 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 362/711] 10.222.0.57 () {56 vars in 1112 bytes} [Wed Aug 10 08:23:56 2022] GET /identity/v3/services?name=network => generated 124 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:57.200886 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719d9e18-06f6-438d-b959-014ae2b065b8 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:57.255344 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed87485-3407-48b5-8554-428aed1549bd None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:57.290369 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d47baeb-9b1e-43b9-bd22-c2857b26153b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:57.291008 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d47baeb-9b1e-43b9-bd22-c2857b26153b None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:57.291292 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d47baeb-9b1e-43b9-bd22-c2857b26153b None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:57.291568 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d47baeb-9b1e-43b9-bd22-c2857b26153b None admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:57.292306 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d47baeb-9b1e-43b9-bd22-c2857b26153b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:57.292615 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d47baeb-9b1e-43b9-bd22-c2857b26153b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:57.294165 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d47baeb-9b1e-43b9-bd22-c2857b26153b None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:57.317093 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 350/712] 10.222.0.57 () {56 vars in 1133 bytes} [Wed Aug 10 08:23:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:57.319354 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d47baeb-9b1e-43b9-bd22-c2857b26153b None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:57.339507 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:57.371431 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719d9e18-06f6-438d-b959-014ae2b065b8 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:57.443467 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:57.448881 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76746dca-366d-4e26-8881-a3752158da6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:57.449517 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76746dca-366d-4e26-8881-a3752158da6e None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:57.449798 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76746dca-366d-4e26-8881-a3752158da6e None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:57.450070 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76746dca-366d-4e26-8881-a3752158da6e None admin] PATH_INFO: `/v3/services` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:57.450701 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76746dca-366d-4e26-8881-a3752158da6e None admin] RBAC: Adding query filter params (type=network) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:57.451002 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76746dca-366d-4e26-8881-a3752158da6e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:57.452487 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76746dca-366d-4e26-8881-a3752158da6e None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:57.465299 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 363/713] 10.222.0.57 () {56 vars in 1112 bytes} [Wed Aug 10 08:23:57 2022] GET /identity/v3/services?type=network => generated 124 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:57.473784 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d47baeb-9b1e-43b9-bd22-c2857b26153b None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:57.523710 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9f392a-372d-43cf-a502-ed907bae7e87 None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:57.523996 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29282a8a-7685-4885-832a-775f02a298b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:57.524125 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9f392a-372d-43cf-a502-ed907bae7e87 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:57.524275 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9f392a-372d-43cf-a502-ed907bae7e87 None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:57.524612 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29282a8a-7685-4885-832a-775f02a298b4 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:57.524890 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29282a8a-7685-4885-832a-775f02a298b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:57.525182 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29282a8a-7685-4885-832a-775f02a298b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:57.525650 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 364/714] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:23:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:23:57.531496 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4302a87d-943a-45e6-bd1f-bd3275591f01 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:57.531785 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4302a87d-943a-45e6-bd1f-bd3275591f01 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:57.532067 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4302a87d-943a-45e6-bd1f-bd3275591f01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:57.574842 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-4302a87d-943a-45e6-bd1f-bd3275591f01 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:23:57.579911 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-4302a87d-943a-45e6-bd1f-bd3275591f01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:23:57.654963 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29282a8a-7685-4885-832a-775f02a298b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:57.657901 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29282a8a-7685-4885-832a-775f02a298b4 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:57.703302 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 365/715] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:23:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 08:23:57.732120 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4302a87d-943a-45e6-bd1f-bd3275591f01 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:57.765282 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:57.783588 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 351/716] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Aug 10 08:23:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 10 08:23:57.801683 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29282a8a-7685-4885-832a-775f02a298b4 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:57.827941 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:57.844462 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4302a87d-943a-45e6-bd1f-bd3275591f01 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:57.882511 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f38dcf5-e425-4d39-bba7-1aaf370dbf9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:57.883462 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f38dcf5-e425-4d39-bba7-1aaf370dbf9e None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:57.883925 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f38dcf5-e425-4d39-bba7-1aaf370dbf9e None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:57.884374 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f38dcf5-e425-4d39-bba7-1aaf370dbf9e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:57.896123 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f38dcf5-e425-4d39-bba7-1aaf370dbf9e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 08:23:57.901515 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-1f38dcf5-e425-4d39-bba7-1aaf370dbf9e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 08:23:57.901515 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:57.901515 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:57.901515 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:57.901515 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:57.901515 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:57.901515 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:57.901515 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:57.901515 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:57.901515 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:57.901515 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:57.901515 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:57.901515 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 08:23:57.901515 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 08:23:57.901515 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 08:23:57.901515 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 08:23:57.902493 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 08:23:57.902493 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 08:23:57.902493 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 08:23:57.902493 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 08:23:57.902493 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:57.902493 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:57.902493 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:57.902493 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:57.902493 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:57.902493 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:57.902493 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:57.902493 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:57.902493 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:57.902493 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:57.902493 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:57.902493 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:57.903407 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:57.903407 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:57.903407 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:57.903407 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:57.903407 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 08:23:57.903407 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 08:23:57.903407 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 08:23:57.903407 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 08:23:57.903407 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 08:23:57.903407 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 08:23:57.903407 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 08:23:57.903407 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:23:57.906323 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 366/717] 10.222.0.57 () {56 vars in 1102 bytes} [Wed Aug 10 08:23:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 08:23:57.913741 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f38dcf5-e425-4d39-bba7-1aaf370dbf9e None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:58.000954 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29282a8a-7685-4885-832a-775f02a298b4 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:58.041274 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3691c6f6-39e5-4cd5-acd7-6650edaf9d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:58.041907 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3691c6f6-39e5-4cd5-acd7-6650edaf9d66 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:58.042183 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3691c6f6-39e5-4cd5-acd7-6650edaf9d66 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:58.042448 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3691c6f6-39e5-4cd5-acd7-6650edaf9d66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:58.053841 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3691c6f6-39e5-4cd5-acd7-6650edaf9d66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:58.055330 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3691c6f6-39e5-4cd5-acd7-6650edaf9d66 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:58.077855 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-3691c6f6-39e5-4cd5-acd7-6650edaf9d66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:23:58.077855 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:58.077855 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:58.077855 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:58.077855 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:58.077855 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:58.077855 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:58.077855 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:58.077855 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:58.077855 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:58.077855 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:58.077855 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:58.077855 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 08:23:58.077855 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 08:23:58.077855 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 08:23:58.077855 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 08:23:58.078873 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:58.078873 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:58.078873 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:58.078873 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:58.078873 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:58.078873 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:58.078873 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:58.078873 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:58.078873 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:58.078873 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:58.078873 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:58.078873 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:58.078873 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:58.078873 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:58.078873 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:58.078873 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:58.078873 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 08:23:58.079766 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 08:23:58.079766 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:23:58.079766 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:23:58.082009 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 352/718] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:23:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 08:23:58.083439 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54443f0-512a-4ace-9dee-ceb12b1141b7 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:58.083934 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54443f0-512a-4ace-9dee-ceb12b1141b7 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:58.084403 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54443f0-512a-4ace-9dee-ceb12b1141b7 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:58.085950 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 353/719] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:23:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:23:58.091559 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54443f0-512a-4ace-9dee-ceb12b1141b7 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:58.130686 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:58.168341 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f38dcf5-e425-4d39-bba7-1aaf370dbf9e None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:58.200808 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54443f0-512a-4ace-9dee-ceb12b1141b7 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:58.207653 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f636288-5e69-4e89-ac14-707f2a92c7f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:58.208718 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f636288-5e69-4e89-ac14-707f2a92c7f9 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:58.209230 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f636288-5e69-4e89-ac14-707f2a92c7f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:58.209734 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f636288-5e69-4e89-ac14-707f2a92c7f9 None admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:58.210602 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f636288-5e69-4e89-ac14-707f2a92c7f9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:58.211170 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f636288-5e69-4e89-ac14-707f2a92c7f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:58.212848 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f636288-5e69-4e89-ac14-707f2a92c7f9 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:58.223300 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 367/720] 10.222.0.57 () {56 vars in 1103 bytes} [Wed Aug 10 08:23:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:23:58.225047 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af6f7fc-2904-4841-8d38-dad03fedd55b None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:58.225540 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af6f7fc-2904-4841-8d38-dad03fedd55b None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:58.226017 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af6f7fc-2904-4841-8d38-dad03fedd55b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:58.244744 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c62326-19dc-46b0-90bc-460e919c3fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:58.245630 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c62326-19dc-46b0-90bc-460e919c3fb3 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:58.245909 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c62326-19dc-46b0-90bc-460e919c3fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:58.246176 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c62326-19dc-46b0-90bc-460e919c3fb3 None admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:58.246819 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c62326-19dc-46b0-90bc-460e919c3fb3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:58.247123 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c62326-19dc-46b0-90bc-460e919c3fb3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:58.247437 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-6af6f7fc-2904-4841-8d38-dad03fedd55b None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:23:58.248481 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c62326-19dc-46b0-90bc-460e919c3fb3 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:58.249091 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-6af6f7fc-2904-4841-8d38-dad03fedd55b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:23:58.251503 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:58.267533 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 354/721] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Aug 10 08:23:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:58.267533 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c62326-19dc-46b0-90bc-460e919c3fb3 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:58.349007 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 368/722] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:23:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 08:23:58.350304 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af6f7fc-2904-4841-8d38-dad03fedd55b None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:58.393835 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:58.404570 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c62326-19dc-46b0-90bc-460e919c3fb3 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:58.463242 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31888291-3997-4c1f-b8d1-58888c624ae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:58.463777 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31888291-3997-4c1f-b8d1-58888c624ae3 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:58.464055 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31888291-3997-4c1f-b8d1-58888c624ae3 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:58.464318 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31888291-3997-4c1f-b8d1-58888c624ae3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:58.475627 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31888291-3997-4c1f-b8d1-58888c624ae3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:58.477097 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31888291-3997-4c1f-b8d1-58888c624ae3 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:58.484441 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af6f7fc-2904-4841-8d38-dad03fedd55b None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:58.492073 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-31888291-3997-4c1f-b8d1-58888c624ae3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:23:58.492073 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:58.492073 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:58.492073 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:58.492073 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:58.492073 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:58.492073 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:58.492073 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:58.492073 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:58.492073 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:58.492073 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:58.492073 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:58.492073 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 08:23:58.492073 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 08:23:58.492073 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 08:23:58.492073 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 08:23:58.498062 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:58.498062 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:58.498062 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:58.498062 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:58.498062 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:58.498062 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:58.498062 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:58.498062 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:58.498062 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:58.498062 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:58.498062 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:58.498062 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:58.498062 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:58.498062 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:58.498062 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:58.498062 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:58.498062 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 08:23:58.499214 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 08:23:58.499214 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:23:58.499214 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:23:58.499214 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 355/723] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:23:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:58.499214 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31888291-3997-4c1f-b8d1-58888c624ae3 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:58.527033 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e51b0f3-f368-4596-bf0e-6c31ea25f5a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:58.527651 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e51b0f3-f368-4596-bf0e-6c31ea25f5a6 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:58.527931 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e51b0f3-f368-4596-bf0e-6c31ea25f5a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:58.528193 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e51b0f3-f368-4596-bf0e-6c31ea25f5a6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:58.535744 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e51b0f3-f368-4596-bf0e-6c31ea25f5a6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:58.537205 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e51b0f3-f368-4596-bf0e-6c31ea25f5a6 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:58.538921 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:58.551571 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-4e51b0f3-f368-4596-bf0e-6c31ea25f5a6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 08:23:58.551571 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:58.551571 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:58.551571 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:58.551571 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:58.551571 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:58.551571 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:58.551571 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:58.551571 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:58.551571 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:58.551571 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:58.551571 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:58.551571 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 08:23:58.551571 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 08:23:58.551571 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 08:23:58.552488 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 08:23:58.552488 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:58.552488 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:58.552488 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:58.552488 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:58.552488 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:58.552488 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:58.552488 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:58.552488 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:58.552488 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:58.552488 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:58.552488 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:58.552488 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:58.552488 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:58.552488 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:58.552488 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:58.552488 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:58.553754 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 08:23:58.553754 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 08:23:58.553754 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 08:23:58.553754 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 08:23:58.553754 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 08:23:58.553754 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 08:23:58.553754 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 08:23:58.553754 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:23:58.557480 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 369/724] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Aug 10 08:23:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:58.558532 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cf1a44-a539-46e9-9762-890eb0a5bc3d None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:58.558819 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cf1a44-a539-46e9-9762-890eb0a5bc3d None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:58.559079 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cf1a44-a539-46e9-9762-890eb0a5bc3d None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:58.564205 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 370/725] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:23:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:23:58.564205 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cf1a44-a539-46e9-9762-890eb0a5bc3d None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:58.609568 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31888291-3997-4c1f-b8d1-58888c624ae3 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:58.645119 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d591fa7-51ab-48bb-b2dd-23cb53ee11fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:58.645851 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d591fa7-51ab-48bb-b2dd-23cb53ee11fb None admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:58.646200 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d591fa7-51ab-48bb-b2dd-23cb53ee11fb None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:58.646540 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d591fa7-51ab-48bb-b2dd-23cb53ee11fb None admin] PATH_INFO: `/v3/services` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:58.647250 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d591fa7-51ab-48bb-b2dd-23cb53ee11fb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:58.648690 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d591fa7-51ab-48bb-b2dd-23cb53ee11fb None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:58.671988 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cf1a44-a539-46e9-9762-890eb0a5bc3d None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:58.705731 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5771d66f-574c-4a74-b71d-08f4790c8d0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:58.707382 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5771d66f-574c-4a74-b71d-08f4790c8d0c None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:58.707670 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5771d66f-574c-4a74-b71d-08f4790c8d0c None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:58.707946 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5771d66f-574c-4a74-b71d-08f4790c8d0c None admin] PATH_INFO: `/v3/domains` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:58.708603 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5771d66f-574c-4a74-b71d-08f4790c8d0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:58.708908 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5771d66f-574c-4a74-b71d-08f4790c8d0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:58.710301 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5771d66f-574c-4a74-b71d-08f4790c8d0c None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:58.724652 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 371/726] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Aug 10 08:23:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:58.724652 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5771d66f-574c-4a74-b71d-08f4790c8d0c None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:58.731022 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 356/727] 10.222.0.57 () {60 vars in 1141 bytes} [Wed Aug 10 08:23:58 2022] POST /identity/v3/services => generated 251 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:23:58.732216 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffc30bf-a996-44c8-884c-377a71c26330 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:58.732937 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffc30bf-a996-44c8-884c-377a71c26330 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:58.732937 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffc30bf-a996-44c8-884c-377a71c26330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:58.754597 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-fffc30bf-a996-44c8-884c-377a71c26330 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:23:58.755835 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-fffc30bf-a996-44c8-884c-377a71c26330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:23:58.785344 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:58.817768 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:58.857374 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 357/728] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:23:58 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 08:23:58.858288 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffc30bf-a996-44c8-884c-377a71c26330 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:58.888415 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5771d66f-574c-4a74-b71d-08f4790c8d0c None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:58.929320 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdc10bd-2506-4310-a8a6-b58bada03620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:58.930100 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdc10bd-2506-4310-a8a6-b58bada03620 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:58.930309 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdc10bd-2506-4310-a8a6-b58bada03620 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:58.930529 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdc10bd-2506-4310-a8a6-b58bada03620 None admin] PATH_INFO: `/v3/projects` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:58.931246 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdc10bd-2506-4310-a8a6-b58bada03620 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:58.931552 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdc10bd-2506-4310-a8a6-b58bada03620 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:58.933973 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdc10bd-2506-4310-a8a6-b58bada03620 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:58.948792 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 372/729] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Aug 10 08:23:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:58.950123 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdc10bd-2506-4310-a8a6-b58bada03620 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:59.069480 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdc10bd-2506-4310-a8a6-b58bada03620 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:59.093443 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:59.101928 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffc30bf-a996-44c8-884c-377a71c26330 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:59.106476 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a645e8-24d7-4b39-bb79-c16f708f5d0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:59.107316 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a645e8-24d7-4b39-bb79-c16f708f5d0c None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:59.107788 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a645e8-24d7-4b39-bb79-c16f708f5d0c None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:59.108208 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a645e8-24d7-4b39-bb79-c16f708f5d0c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:59.121640 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a645e8-24d7-4b39-bb79-c16f708f5d0c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 08:23:59.126929 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-33a645e8-24d7-4b39-bb79-c16f708f5d0c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 08:23:59.126929 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:59.126929 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:59.126929 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:59.126929 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:59.126929 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:59.126929 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:59.126929 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:59.126929 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:59.126929 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:59.126929 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:59.126929 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:59.126929 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 08:23:59.126929 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 08:23:59.126929 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 08:23:59.126929 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 08:23:59.128640 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 08:23:59.128640 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 08:23:59.128640 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 08:23:59.128640 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 08:23:59.128640 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:59.128640 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:59.128640 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:59.128640 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:59.128640 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:59.128640 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:59.128640 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:59.128640 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:59.128640 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:59.128640 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:59.128640 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:59.128640 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:59.129622 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:59.129622 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:59.129622 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:59.129622 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:59.129622 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 08:23:59.129622 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 08:23:59.129622 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 08:23:59.129622 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 08:23:59.129622 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 08:23:59.129622 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 08:23:59.129622 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 08:23:59.129622 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:23:59.137041 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 373/730] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Aug 10 08:23:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:59.138247 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a645e8-24d7-4b39-bb79-c16f708f5d0c None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:59.138987 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836d808b-125a-40a1-9fc5-1e1aeb96fbc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:59.139623 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836d808b-125a-40a1-9fc5-1e1aeb96fbc7 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:59.139900 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836d808b-125a-40a1-9fc5-1e1aeb96fbc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:59.140167 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836d808b-125a-40a1-9fc5-1e1aeb96fbc7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:59.142163 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:59.148078 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836d808b-125a-40a1-9fc5-1e1aeb96fbc7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:59.149564 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836d808b-125a-40a1-9fc5-1e1aeb96fbc7 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:59.161402 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-836d808b-125a-40a1-9fc5-1e1aeb96fbc7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 08:23:59.161402 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:59.161402 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:59.161402 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:59.161402 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:59.161402 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:59.161402 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:59.161402 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:59.161402 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:59.161402 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:59.161402 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:59.161402 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:59.161402 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 08:23:59.161402 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 08:23:59.161402 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 08:23:59.161402 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 08:23:59.162619 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:59.162619 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:59.162619 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 08:23:59.162619 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 08:23:59.162619 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 08:23:59.162619 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 08:23:59.162619 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:59.162619 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:59.162619 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:59.162619 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:59.162619 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:59.162619 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:59.162619 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:59.162619 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:59.162619 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:59.162619 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:59.162619 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:59.162619 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:59.163906 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:59.163906 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:59.163906 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 08:23:59.163906 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 08:23:59.163906 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 08:23:59.163906 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 08:23:59.163906 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 08:23:59.163906 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 08:23:59.163906 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 08:23:59.163906 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:23:59.167070 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 358/731] 10.222.0.57 () {56 vars in 1117 bytes} [Wed Aug 10 08:23:58 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:59.168129 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836d808b-125a-40a1-9fc5-1e1aeb96fbc7 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:59.255003 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a645e8-24d7-4b39-bb79-c16f708f5d0c None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:59.276489 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836d808b-125a-40a1-9fc5-1e1aeb96fbc7 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:59.293677 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28030cbe-110c-4d5c-a810-399db1ad800c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:59.294356 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28030cbe-110c-4d5c-a810-399db1ad800c None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:59.294639 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28030cbe-110c-4d5c-a810-399db1ad800c None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:59.294910 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28030cbe-110c-4d5c-a810-399db1ad800c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:59.295636 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28030cbe-110c-4d5c-a810-399db1ad800c None admin] RBAC: Adding query filter params (role.id=ec3751692d9c4ecf81b4a71c936c46be, scope.project.id=b49e381419e940278bcb6111899589b5, user.id=a943008193914f76a663d489bd73e873) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:59.295944 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28030cbe-110c-4d5c-a810-399db1ad800c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:59.297427 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28030cbe-110c-4d5c-a810-399db1ad800c None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:59.303572 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 374/732] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Aug 10 08:23:59 2022] GET /identity/v3/role_assignments?user.id=a943008193914f76a663d489bd73e873&scope.project.id=b49e381419e940278bcb6111899589b5&role.id=ec3751692d9c4ecf81b4a71c936c46be => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:59.304448 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28030cbe-110c-4d5c-a810-399db1ad800c None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:59.318387 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f7b2c2-28b6-464d-92bd-256fcecb6295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:59.319394 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f7b2c2-28b6-464d-92bd-256fcecb6295 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:59.319946 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f7b2c2-28b6-464d-92bd-256fcecb6295 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:59.320501 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f7b2c2-28b6-464d-92bd-256fcecb6295 None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:59.321527 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f7b2c2-28b6-464d-92bd-256fcecb6295 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:59.322064 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f7b2c2-28b6-464d-92bd-256fcecb6295 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:59.323686 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f7b2c2-28b6-464d-92bd-256fcecb6295 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:59.335015 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 359/733] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Aug 10 08:23:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:59.343800 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f7b2c2-28b6-464d-92bd-256fcecb6295 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:59.362071 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:59.420304 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28030cbe-110c-4d5c-a810-399db1ad800c None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:59.459996 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe845e86-10bb-4b65-805f-17ced95e0d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:59.460623 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe845e86-10bb-4b65-805f-17ced95e0d61 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:59.460897 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe845e86-10bb-4b65-805f-17ced95e0d61 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:59.461266 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe845e86-10bb-4b65-805f-17ced95e0d61 None admin] PATH_INFO: `/v3/users` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:59.462011 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe845e86-10bb-4b65-805f-17ced95e0d61 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:59.462317 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe845e86-10bb-4b65-805f-17ced95e0d61 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:59.463850 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe845e86-10bb-4b65-805f-17ced95e0d61 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:59.483888 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 375/734] 10.222.0.57 () {56 vars in 1149 bytes} [Wed Aug 10 08:23:59 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:59.486415 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:59.489735 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe845e86-10bb-4b65-805f-17ced95e0d61 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:59.563905 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f7b2c2-28b6-464d-92bd-256fcecb6295 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:59.602227 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd4ae1b-322e-4064-91a8-36d5ff194c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:59.603302 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd4ae1b-322e-4064-91a8-36d5ff194c09 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:59.603807 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd4ae1b-322e-4064-91a8-36d5ff194c09 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:59.604296 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd4ae1b-322e-4064-91a8-36d5ff194c09 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:59.616186 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd4ae1b-322e-4064-91a8-36d5ff194c09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:59.618223 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd4ae1b-322e-4064-91a8-36d5ff194c09 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:59.634748 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-3bd4ae1b-322e-4064-91a8-36d5ff194c09 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:23:59.634748 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:23:59.634748 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:23:59.634748 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:23:59.634748 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:23:59.634748 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:23:59.634748 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:23:59.634748 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:23:59.634748 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:23:59.634748 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:23:59.634748 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:23:59.634748 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:23:59.634748 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 08:23:59.634748 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 08:23:59.634748 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 08:23:59.634748 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 08:23:59.635797 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:23:59.635797 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:23:59.635797 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:23:59.635797 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:23:59.635797 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:23:59.635797 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:23:59.635797 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:23:59.635797 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:23:59.635797 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:23:59.635797 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:23:59.635797 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:23:59.635797 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:23:59.635797 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:23:59.635797 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:23:59.635797 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:23:59.635797 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:23:59.635797 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 08:23:59.636709 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 08:23:59.636709 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:23:59.636709 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:23:59.639863 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 360/735] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:23:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:23:59.640873 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:59.645778 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd4ae1b-322e-4064-91a8-36d5ff194c09 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:59.646706 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe845e86-10bb-4b65-805f-17ced95e0d61 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:59.719906 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb774a30-7342-4472-8a0c-39cd5451ca1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:59.720533 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb774a30-7342-4472-8a0c-39cd5451ca1a None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:59.720836 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb774a30-7342-4472-8a0c-39cd5451ca1a None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:59.721171 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb774a30-7342-4472-8a0c-39cd5451ca1a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:59.759044 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd4ae1b-322e-4064-91a8-36d5ff194c09 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:59.770152 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:59.802161 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e77d03-25c6-496f-a120-ae0f5b7419db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:23:59.803077 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e77d03-25c6-496f-a120-ae0f5b7419db None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:59.803408 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e77d03-25c6-496f-a120-ae0f5b7419db None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:59.803686 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e77d03-25c6-496f-a120-ae0f5b7419db None admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:59.804332 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e77d03-25c6-496f-a120-ae0f5b7419db None admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:23:59.804636 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e77d03-25c6-496f-a120-ae0f5b7419db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:59.806665 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e77d03-25c6-496f-a120-ae0f5b7419db None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:59.824149 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 361/736] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Aug 10 08:23:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:23:59.824331 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb774a30-7342-4472-8a0c-39cd5451ca1a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:23:59.825631 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf7427f-dc97-4f94-8486-9fddbf27da38 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:59.825971 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf7427f-dc97-4f94-8486-9fddbf27da38 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:59.826297 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf7427f-dc97-4f94-8486-9fddbf27da38 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:59.827543 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 362/737] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:23:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:23:59.830171 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb774a30-7342-4472-8a0c-39cd5451ca1a None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:23:59.834502 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf7427f-dc97-4f94-8486-9fddbf27da38 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:23:59.948312 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:23:59.958510 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 376/738] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Aug 10 08:23:59 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 08:23:59.960600 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b19a33-91bf-4249-95be-23832ab33b0b None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:23:59.960600 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b19a33-91bf-4249-95be-23832ab33b0b None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:23:59.961104 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b19a33-91bf-4249-95be-23832ab33b0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:23:59.968329 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf7427f-dc97-4f94-8486-9fddbf27da38 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:23:59.983880 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-09b19a33-91bf-4249-95be-23832ab33b0b None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:23:59.985401 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-09b19a33-91bf-4249-95be-23832ab33b0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:24:00.011041 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d288c0a-9b12-4748-af2b-84c92f0c4d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:00.012103 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d288c0a-9b12-4748-af2b-84c92f0c4d64 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:00.012615 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d288c0a-9b12-4748-af2b-84c92f0c4d64 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:00.013202 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d288c0a-9b12-4748-af2b-84c92f0c4d64 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:00.021809 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d288c0a-9b12-4748-af2b-84c92f0c4d64 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:00.023635 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d288c0a-9b12-4748-af2b-84c92f0c4d64 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:00.036860 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-3d288c0a-9b12-4748-af2b-84c92f0c4d64 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 10 08:24:00.036860 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:00.036860 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:24:00.036860 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:24:00.036860 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:24:00.036860 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:24:00.036860 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:24:00.036860 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:24:00.036860 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:24:00.036860 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:24:00.036860 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:24:00.036860 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:24:00.036860 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 08:24:00.036860 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 08:24:00.036860 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 08:24:00.036860 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 08:24:00.037898 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:24:00.037898 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:24:00.037898 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 08:24:00.037898 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 08:24:00.037898 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 08:24:00.037898 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 08:24:00.037898 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:24:00.037898 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:24:00.037898 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:24:00.037898 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:24:00.037898 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:24:00.037898 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:24:00.037898 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:24:00.037898 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:24:00.037898 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:24:00.037898 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:24:00.037898 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:24:00.037898 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:24:00.038873 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:24:00.038873 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:24:00.038873 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 08:24:00.038873 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 08:24:00.038873 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 08:24:00.038873 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 08:24:00.038873 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 08:24:00.038873 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 08:24:00.038873 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 10 08:24:00.038873 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:24:00.046558 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 363/739] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Aug 10 08:23:59 2022] GET /identity/v3/users/swift => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:24:00.047848 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d288c0a-9b12-4748-af2b-84c92f0c4d64 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:00.065072 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 377/740] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:23:59 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 08:24:00.066737 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b19a33-91bf-4249-95be-23832ab33b0b None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:00.079708 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:00.164483 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d288c0a-9b12-4748-af2b-84c92f0c4d64 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:00.194474 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b19a33-91bf-4249-95be-23832ab33b0b None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:00.203224 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d0c876-0ce3-4b86-ba00-2d1a7d348d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:00.204219 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d0c876-0ce3-4b86-ba00-2d1a7d348d17 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:00.204706 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d0c876-0ce3-4b86-ba00-2d1a7d348d17 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:00.205305 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d0c876-0ce3-4b86-ba00-2d1a7d348d17 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:00.217080 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d0c876-0ce3-4b86-ba00-2d1a7d348d17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:00.218997 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d0c876-0ce3-4b86-ba00-2d1a7d348d17 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:00.224563 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:00.230225 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f49fc74-903f-4d5b-a68e-b6ed36ecccee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:00.230836 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f49fc74-903f-4d5b-a68e-b6ed36ecccee None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:00.231111 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f49fc74-903f-4d5b-a68e-b6ed36ecccee None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:00.231377 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f49fc74-903f-4d5b-a68e-b6ed36ecccee None admin] PATH_INFO: `/v3/users` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:00.232096 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f49fc74-903f-4d5b-a68e-b6ed36ecccee None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:00.232395 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f49fc74-903f-4d5b-a68e-b6ed36ecccee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:00.233843 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f49fc74-903f-4d5b-a68e-b6ed36ecccee None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:00.238361 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-53d0c876-0ce3-4b86-ba00-2d1a7d348d17 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:24:00.238361 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:00.238361 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:24:00.238361 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:24:00.238361 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:24:00.238361 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:24:00.238361 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:24:00.238361 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:24:00.238361 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:24:00.238361 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:24:00.238361 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:24:00.238361 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:24:00.238361 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 08:24:00.238361 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 08:24:00.238361 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 08:24:00.238361 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 08:24:00.245610 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:24:00.245610 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:24:00.245610 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:24:00.245610 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:24:00.245610 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:24:00.245610 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:24:00.245610 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:24:00.245610 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:24:00.245610 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:24:00.245610 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:24:00.245610 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:24:00.245610 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:24:00.245610 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:24:00.245610 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:24:00.245610 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:24:00.245610 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:24:00.245610 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 08:24:00.246533 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 08:24:00.246533 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:24:00.246533 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:24:00.247272 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 364/741] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:24:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:24:00.248305 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d0c876-0ce3-4b86-ba00-2d1a7d348d17 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:00.258867 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 378/742] 10.222.0.57 () {56 vars in 1135 bytes} [Wed Aug 10 08:24:00 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:00.258867 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f49fc74-903f-4d5b-a68e-b6ed36ecccee None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:00.342456 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:00.364303 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d0c876-0ce3-4b86-ba00-2d1a7d348d17 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:00.365547 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f49fc74-903f-4d5b-a68e-b6ed36ecccee None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:00.410592 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cebc9c-98df-4519-a461-50a469e02d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:00.411209 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cebc9c-98df-4519-a461-50a469e02d0b None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:00.411486 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cebc9c-98df-4519-a461-50a469e02d0b None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:00.411755 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cebc9c-98df-4519-a461-50a469e02d0b None admin] PATH_INFO: `/v3/domains` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:00.412431 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cebc9c-98df-4519-a461-50a469e02d0b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:00.412754 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cebc9c-98df-4519-a461-50a469e02d0b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:00.414172 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d52e33-5604-4d96-8170-492d0b164a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:00.414300 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cebc9c-98df-4519-a461-50a469e02d0b None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:00.417304 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d52e33-5604-4d96-8170-492d0b164a82 None admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:00.417304 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d52e33-5604-4d96-8170-492d0b164a82 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:00.417304 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d52e33-5604-4d96-8170-492d0b164a82 None admin] PATH_INFO: `/v3/services` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:00.417304 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d52e33-5604-4d96-8170-492d0b164a82 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:00.418576 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d52e33-5604-4d96-8170-492d0b164a82 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:00.431381 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 379/743] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Aug 10 08:24:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:00.431381 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cebc9c-98df-4519-a461-50a469e02d0b None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:00.483126 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:00.523774 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 365/744] 10.222.0.57 () {60 vars in 1141 bytes} [Wed Aug 10 08:24:00 2022] POST /identity/v3/services => generated 240 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:24:00.525439 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d52e33-5604-4d96-8170-492d0b164a82 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:00.562940 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cebc9c-98df-4519-a461-50a469e02d0b None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:00.654509 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b20d5dd-029c-4ef5-881d-b6c9e2d0a5fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:00.655255 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b20d5dd-029c-4ef5-881d-b6c9e2d0a5fc None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:00.655538 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b20d5dd-029c-4ef5-881d-b6c9e2d0a5fc None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:00.655807 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b20d5dd-029c-4ef5-881d-b6c9e2d0a5fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:00.746401 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d52e33-5604-4d96-8170-492d0b164a82 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:00.768178 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b20d5dd-029c-4ef5-881d-b6c9e2d0a5fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:00.772061 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b20d5dd-029c-4ef5-881d-b6c9e2d0a5fc None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:00.796705 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9208a568-4748-4453-a8d2-c57f283c455a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:00.798246 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9208a568-4748-4453-a8d2-c57f283c455a None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:00.798964 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9208a568-4748-4453-a8d2-c57f283c455a None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:00.800347 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9208a568-4748-4453-a8d2-c57f283c455a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:00.812036 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9208a568-4748-4453-a8d2-c57f283c455a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:00.815009 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9208a568-4748-4453-a8d2-c57f283c455a None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:00.821835 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:00.839414 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-9208a568-4748-4453-a8d2-c57f283c455a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 08:24:00.839414 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:00.839414 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:24:00.839414 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:24:00.839414 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:24:00.839414 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:24:00.839414 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:24:00.839414 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:24:00.839414 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:24:00.839414 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:24:00.839414 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:24:00.839414 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:24:00.839414 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 08:24:00.839414 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 08:24:00.839414 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 08:24:00.842038 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 08:24:00.842038 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:24:00.842038 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:24:00.842038 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:24:00.842038 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:24:00.842038 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:24:00.842038 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:24:00.842038 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:24:00.842038 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:24:00.842038 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:24:00.842038 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:24:00.842038 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:24:00.842038 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:24:00.842038 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:24:00.842038 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:24:00.842038 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:24:00.842038 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:24:00.843799 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 08:24:00.843799 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 08:24:00.843799 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 08:24:00.843799 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 08:24:00.843799 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 08:24:00.843799 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 08:24:00.843799 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 08:24:00.843799 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:24:00.847059 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 366/745] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Aug 10 08:24:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:24:00.855768 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:00.857021 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9208a568-4748-4453-a8d2-c57f283c455a None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:00.972527 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 380/746] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Aug 10 08:24:00 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 545 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 08:24:00.983577 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b20d5dd-029c-4ef5-881d-b6c9e2d0a5fc None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:00.997782 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9208a568-4748-4453-a8d2-c57f283c455a None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:01.045273 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c3bbed-a45f-4683-813c-1811102d1d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:01.045882 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c3bbed-a45f-4683-813c-1811102d1d2b None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:01.046156 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c3bbed-a45f-4683-813c-1811102d1d2b None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:01.047846 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c3bbed-a45f-4683-813c-1811102d1d2b None admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:01.048561 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c3bbed-a45f-4683-813c-1811102d1d2b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:01.048868 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c3bbed-a45f-4683-813c-1811102d1d2b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:01.051043 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c3bbed-a45f-4683-813c-1811102d1d2b None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:01.065213 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 367/747] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Aug 10 08:24:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:24:01.067106 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d762d966-2fc7-45ff-8f9f-ad3e232945ca None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:01.067393 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d762d966-2fc7-45ff-8f9f-ad3e232945ca None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:01.067668 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d762d966-2fc7-45ff-8f9f-ad3e232945ca None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:01.072957 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 368/748] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:24:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:24:01.073823 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d762d966-2fc7-45ff-8f9f-ad3e232945ca None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:01.146479 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:01.155757 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b20d5dd-029c-4ef5-881d-b6c9e2d0a5fc None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:01.199771 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d762d966-2fc7-45ff-8f9f-ad3e232945ca None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:01.203394 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755f57bf-ccb3-46e0-be6d-78f2894ee4be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:01.204222 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755f57bf-ccb3-46e0-be6d-78f2894ee4be None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:01.204643 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755f57bf-ccb3-46e0-be6d-78f2894ee4be None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:01.205116 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755f57bf-ccb3-46e0-be6d-78f2894ee4be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:01.220014 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755f57bf-ccb3-46e0-be6d-78f2894ee4be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:01.222654 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755f57bf-ccb3-46e0-be6d-78f2894ee4be None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:01.226439 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:01.242483 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-755f57bf-ccb3-46e0-be6d-78f2894ee4be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:24:01.242483 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:01.242483 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:24:01.242483 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:24:01.242483 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:24:01.242483 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:24:01.242483 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:24:01.242483 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:24:01.242483 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:24:01.242483 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:24:01.242483 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:24:01.242483 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:24:01.242483 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 08:24:01.242483 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 08:24:01.242483 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 08:24:01.242483 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 08:24:01.243582 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:24:01.243582 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:24:01.243582 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:24:01.243582 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:24:01.243582 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:24:01.243582 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:24:01.243582 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:24:01.243582 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:24:01.243582 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:24:01.243582 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:24:01.243582 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:24:01.243582 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:24:01.243582 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:24:01.243582 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:24:01.243582 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:24:01.243582 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:24:01.243582 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 08:24:01.244997 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 08:24:01.244997 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:24:01.244997 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:24:01.244997 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 381/749] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:24:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:24:01.246518 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95c82db-4b59-4a3a-99a4-4d9f30cbcde9 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:01.246859 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95c82db-4b59-4a3a-99a4-4d9f30cbcde9 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:01.247195 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95c82db-4b59-4a3a-99a4-4d9f30cbcde9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:01.247395 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563616e1-7905-42e5-96ee-c0160c947068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:01.248233 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563616e1-7905-42e5-96ee-c0160c947068 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:01.248777 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563616e1-7905-42e5-96ee-c0160c947068 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:01.249230 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563616e1-7905-42e5-96ee-c0160c947068 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:01.250123 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563616e1-7905-42e5-96ee-c0160c947068 None admin] RBAC: Adding query filter params (role.id=73fdd60e2f804ee69b76fa9145a472ff, scope.project.id=b49e381419e940278bcb6111899589b5, user.id=c140d31bd2b84c48a170e3ca90a041ce) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:01.250592 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563616e1-7905-42e5-96ee-c0160c947068 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:01.252171 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563616e1-7905-42e5-96ee-c0160c947068 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:01.259722 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 369/750] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Aug 10 08:24:01 2022] GET /identity/v3/role_assignments?user.id=c140d31bd2b84c48a170e3ca90a041ce&scope.project.id=b49e381419e940278bcb6111899589b5&role.id=73fdd60e2f804ee69b76fa9145a472ff => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:01.260858 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563616e1-7905-42e5-96ee-c0160c947068 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:01.285563 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-a95c82db-4b59-4a3a-99a4-4d9f30cbcde9 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:24:01.286159 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-a95c82db-4b59-4a3a-99a4-4d9f30cbcde9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:24:01.383456 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563616e1-7905-42e5-96ee-c0160c947068 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:01.424387 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:01.428412 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fb022b-224d-4132-9e8c-12b28318e42c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:01.429360 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fb022b-224d-4132-9e8c-12b28318e42c None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:01.429968 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fb022b-224d-4132-9e8c-12b28318e42c None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:01.430564 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fb022b-224d-4132-9e8c-12b28318e42c None admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:01.431649 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fb022b-224d-4132-9e8c-12b28318e42c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:01.432215 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fb022b-224d-4132-9e8c-12b28318e42c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:01.433988 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fb022b-224d-4132-9e8c-12b28318e42c None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:01.448605 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 370/751] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Aug 10 08:24:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:01.454939 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fb022b-224d-4132-9e8c-12b28318e42c None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:01.467996 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 382/752] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:24:01 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 08:24:01.499944 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95c82db-4b59-4a3a-99a4-4d9f30cbcde9 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:01.630665 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fb022b-224d-4132-9e8c-12b28318e42c None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:01.651185 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95c82db-4b59-4a3a-99a4-4d9f30cbcde9 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:01.675869 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2562813b-bd3e-4c38-b800-b9978fd495b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:01.676377 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2562813b-bd3e-4c38-b800-b9978fd495b8 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:01.676652 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2562813b-bd3e-4c38-b800-b9978fd495b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:01.677005 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2562813b-bd3e-4c38-b800-b9978fd495b8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:01.681534 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:01.687079 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2562813b-bd3e-4c38-b800-b9978fd495b8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:01.688626 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2562813b-bd3e-4c38-b800-b9978fd495b8 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:01.691906 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1079f6-88c8-42c3-96f6-7d532d3062ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:01.692776 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1079f6-88c8-42c3-96f6-7d532d3062ca None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:01.693277 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1079f6-88c8-42c3-96f6-7d532d3062ca None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:01.693742 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1079f6-88c8-42c3-96f6-7d532d3062ca None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:01.694550 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1079f6-88c8-42c3-96f6-7d532d3062ca None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:01.696110 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1079f6-88c8-42c3-96f6-7d532d3062ca None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:01.700760 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-2562813b-bd3e-4c38-b800-b9978fd495b8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 08:24:01.700760 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:01.700760 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:24:01.700760 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:24:01.700760 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:24:01.700760 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:24:01.700760 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:24:01.700760 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:24:01.700760 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:24:01.700760 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:24:01.700760 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:24:01.700760 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:24:01.700760 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 08:24:01.700760 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 08:24:01.700760 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 08:24:01.701679 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 08:24:01.701679 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:24:01.701679 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:24:01.701679 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:24:01.701679 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:24:01.701679 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:24:01.701679 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:24:01.701679 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:24:01.701679 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:24:01.701679 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:24:01.701679 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:24:01.701679 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:24:01.701679 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:24:01.701679 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:24:01.701679 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:24:01.701679 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:24:01.701679 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:24:01.703142 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 08:24:01.703142 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 08:24:01.703142 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 08:24:01.703142 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 08:24:01.703142 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 08:24:01.703142 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 08:24:01.703142 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 08:24:01.703142 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:24:01.703142 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 371/753] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Aug 10 08:24:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 08:24:01.703636 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b27ab62-c34d-4079-9ba8-ae18a06e96f6 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:01.703636 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b27ab62-c34d-4079-9ba8-ae18a06e96f6 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:01.703945 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b27ab62-c34d-4079-9ba8-ae18a06e96f6 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:01.709586 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 372/754] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:24:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:24:01.710550 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a99159-e712-4882-bdee-49e600803c6c None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:01.710821 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a99159-e712-4882-bdee-49e600803c6c None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:01.711162 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a99159-e712-4882-bdee-49e600803c6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:01.711736 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-6e1079f6-88c8-42c3-96f6-7d532d3062ca None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 08:24:01.711736 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:01.711736 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 08:24:01.711736 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 08:24:01.711736 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 08:24:01.711736 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 08:24:01.711736 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 08:24:01.711736 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 08:24:01.711736 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 08:24:01.711736 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:24:01.711736 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 08:24:01.711736 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:24:01.711736 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:01.711736 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:24:01.711736 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:24:01.711736 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:24:01.711736 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:24:01.712685 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:24:01.712685 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:24:01.712685 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:24:01.712685 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:24:01.712685 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:24:01.712685 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:24:01.712685 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 08:24:01.712685 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 08:24:01.712685 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 08:24:01.712685 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 08:24:01.712685 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:24:01.712685 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:24:01.712685 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:24:01.712685 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:24:01.712685 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:24:01.712685 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:24:01.713604 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:24:01.713604 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:24:01.713604 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:24:01.713604 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:24:01.713604 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:24:01.713604 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:24:01.713604 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:24:01.713604 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:24:01.713604 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:24:01.713604 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:24:01.713604 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 08:24:01.713604 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 08:24:01.713604 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 08:24:01.713604 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:24:01.716007 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 383/755] 10.222.0.57 () {56 vars in 1129 bytes} [Wed Aug 10 08:24:01 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 08:24:01.717017 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1079f6-88c8-42c3-96f6-7d532d3062ca None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:01.733182 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-14a99159-e712-4882-bdee-49e600803c6c None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:24:01.734507 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-14a99159-e712-4882-bdee-49e600803c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:24:01.789379 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:01.829067 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 373/756] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:24:01 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 08:24:01.830484 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a99159-e712-4882-bdee-49e600803c6c None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:01.859376 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1079f6-88c8-42c3-96f6-7d532d3062ca None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:01.896949 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e69895-e9cf-42ec-883d-85d56e724b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:01.897480 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e69895-e9cf-42ec-883d-85d56e724b0b None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:01.897757 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e69895-e9cf-42ec-883d-85d56e724b0b None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:01.898024 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e69895-e9cf-42ec-883d-85d56e724b0b None admin] PATH_INFO: `/v3/projects` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:01.898764 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e69895-e9cf-42ec-883d-85d56e724b0b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:01.899131 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e69895-e9cf-42ec-883d-85d56e724b0b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:01.900666 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e69895-e9cf-42ec-883d-85d56e724b0b None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:01.915103 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 384/757] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Aug 10 08:24:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:01.916107 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e69895-e9cf-42ec-883d-85d56e724b0b None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:01.949055 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a99159-e712-4882-bdee-49e600803c6c None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:01.969902 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:01.988688 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc011367-efbf-430d-bfaa-09d518c148c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:01.990332 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc011367-efbf-430d-bfaa-09d518c148c9 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:01.990831 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc011367-efbf-430d-bfaa-09d518c148c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:01.991468 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc011367-efbf-430d-bfaa-09d518c148c9 None admin] PATH_INFO: `/v3/services` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:01.992420 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc011367-efbf-430d-bfaa-09d518c148c9 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:01.992875 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc011367-efbf-430d-bfaa-09d518c148c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:01.995785 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc011367-efbf-430d-bfaa-09d518c148c9 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:02.001680 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 374/758] 10.222.0.57 () {56 vars in 1124 bytes} [Wed Aug 10 08:24:01 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:02.004248 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc011367-efbf-430d-bfaa-09d518c148c9 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:02.030967 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e69895-e9cf-42ec-883d-85d56e724b0b None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:02.061809 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:02.070303 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a498dc9c-f8f1-4f95-a8f5-24f41209b454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:02.071053 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a498dc9c-f8f1-4f95-a8f5-24f41209b454 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:02.071479 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a498dc9c-f8f1-4f95-a8f5-24f41209b454 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:02.074455 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a498dc9c-f8f1-4f95-a8f5-24f41209b454 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:02.083760 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a498dc9c-f8f1-4f95-a8f5-24f41209b454 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 08:24:02.089019 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-a498dc9c-f8f1-4f95-a8f5-24f41209b454 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 08:24:02.089019 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:02.089019 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:24:02.089019 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:24:02.089019 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:24:02.089019 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:24:02.089019 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:24:02.089019 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:24:02.089019 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:24:02.089019 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:24:02.089019 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:24:02.089019 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:24:02.089019 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 08:24:02.089019 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 08:24:02.089019 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 08:24:02.089019 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 08:24:02.090163 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 08:24:02.090163 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 08:24:02.090163 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 08:24:02.090163 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 08:24:02.090163 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:24:02.090163 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:24:02.090163 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:24:02.090163 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:24:02.090163 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:24:02.090163 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:24:02.090163 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:24:02.090163 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:24:02.090163 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:24:02.090163 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:24:02.090163 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:24:02.090163 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:24:02.091219 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:24:02.091219 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:24:02.091219 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:24:02.091219 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:24:02.091219 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 08:24:02.091219 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 08:24:02.091219 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 08:24:02.091219 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 08:24:02.091219 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 08:24:02.091219 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 08:24:02.091219 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 08:24:02.091219 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:24:02.098411 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 385/759] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Aug 10 08:24:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:24:02.099334 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a498dc9c-f8f1-4f95-a8f5-24f41209b454 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:02.139876 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc011367-efbf-430d-bfaa-09d518c148c9 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:02.175319 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c50f0d-fbca-4dfd-abd4-d0c26daed1b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:02.176127 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c50f0d-fbca-4dfd-abd4-d0c26daed1b5 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:02.176531 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c50f0d-fbca-4dfd-abd4-d0c26daed1b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:02.176900 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c50f0d-fbca-4dfd-abd4-d0c26daed1b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:02.177755 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c50f0d-fbca-4dfd-abd4-d0c26daed1b5 None admin] RBAC: Adding query filter params (role.id=ec3751692d9c4ecf81b4a71c936c46be, scope.project.id=b49e381419e940278bcb6111899589b5, user.id=b4b26ba3a9e04c38b52db2a46198a53c) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:02.178227 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c50f0d-fbca-4dfd-abd4-d0c26daed1b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:02.179765 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c50f0d-fbca-4dfd-abd4-d0c26daed1b5 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:02.193288 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 375/760] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Aug 10 08:24:02 2022] GET /identity/v3/role_assignments?user.id=b4b26ba3a9e04c38b52db2a46198a53c&scope.project.id=b49e381419e940278bcb6111899589b5&role.id=ec3751692d9c4ecf81b4a71c936c46be => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:02.194499 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c50f0d-fbca-4dfd-abd4-d0c26daed1b5 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:02.226951 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a498dc9c-f8f1-4f95-a8f5-24f41209b454 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:02.240766 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:02.270018 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091d03f4-6d36-44f5-a291-43e95808bfcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:02.270917 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d03f4-6d36-44f5-a291-43e95808bfcf None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:02.271359 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d03f4-6d36-44f5-a291-43e95808bfcf None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:02.271777 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d03f4-6d36-44f5-a291-43e95808bfcf None admin] PATH_INFO: `/v3/services` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:02.272556 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091d03f4-6d36-44f5-a291-43e95808bfcf None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:02.273028 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091d03f4-6d36-44f5-a291-43e95808bfcf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:02.275470 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091d03f4-6d36-44f5-a291-43e95808bfcf None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:02.281916 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 386/761] 10.222.0.57 () {56 vars in 1124 bytes} [Wed Aug 10 08:24:02 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:02.287993 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091d03f4-6d36-44f5-a291-43e95808bfcf None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:02.416954 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091d03f4-6d36-44f5-a291-43e95808bfcf None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:02.451702 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c50f0d-fbca-4dfd-abd4-d0c26daed1b5 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:02.456520 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:02.461011 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7142017b-7bb4-4977-982a-ab9e7bb4906b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:02.461887 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7142017b-7bb4-4977-982a-ab9e7bb4906b None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:02.462318 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7142017b-7bb4-4977-982a-ab9e7bb4906b None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:02.462737 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7142017b-7bb4-4977-982a-ab9e7bb4906b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:02.463608 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7142017b-7bb4-4977-982a-ab9e7bb4906b None admin] RBAC: Adding query filter params (interface=public, service_id=5dc25e63275d40f28710cf71b59f1cf5, region_id=RegionOne) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:02.464095 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7142017b-7bb4-4977-982a-ab9e7bb4906b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:02.466247 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7142017b-7bb4-4977-982a-ab9e7bb4906b None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:02.477102 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 387/762] 10.222.0.57 () {56 vars in 1251 bytes} [Wed Aug 10 08:24:02 2022] GET /identity/v3/endpoints?service_id=5dc25e63275d40f28710cf71b59f1cf5&interface=public®ion_id=RegionOne => generated 194 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:02.482023 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7142017b-7bb4-4977-982a-ab9e7bb4906b None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:02.500958 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd88f4f-ce04-473f-a938-deab1c26e63c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:02.502936 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd88f4f-ce04-473f-a938-deab1c26e63c None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:02.503218 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd88f4f-ce04-473f-a938-deab1c26e63c None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:02.503883 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd88f4f-ce04-473f-a938-deab1c26e63c None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:02.504589 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd88f4f-ce04-473f-a938-deab1c26e63c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:02.504897 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd88f4f-ce04-473f-a938-deab1c26e63c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:02.506620 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd88f4f-ce04-473f-a938-deab1c26e63c None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:02.525492 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 376/763] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Aug 10 08:24:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:02.530506 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd88f4f-ce04-473f-a938-deab1c26e63c None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:02.551723 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:02.657628 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7142017b-7bb4-4977-982a-ab9e7bb4906b None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:02.715199 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd88f4f-ce04-473f-a938-deab1c26e63c None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:02.720669 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b528225-4b90-4be0-b1de-23a882b11902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:02.721375 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b528225-4b90-4be0-b1de-23a882b11902 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:02.721743 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b528225-4b90-4be0-b1de-23a882b11902 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:02.722021 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b528225-4b90-4be0-b1de-23a882b11902 None admin] PATH_INFO: `/v3/services` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:02.722729 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b528225-4b90-4be0-b1de-23a882b11902 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:02.728400 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b528225-4b90-4be0-b1de-23a882b11902 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:02.737259 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 388/764] 10.222.0.57 () {56 vars in 1087 bytes} [Wed Aug 10 08:24:02 2022] GET /identity/v3/services => generated 772 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:02.773055 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c04163-3c8f-4ace-906a-51015fa5d44b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:02.773708 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c04163-3c8f-4ace-906a-51015fa5d44b None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:02.773987 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c04163-3c8f-4ace-906a-51015fa5d44b None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:02.774259 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c04163-3c8f-4ace-906a-51015fa5d44b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:02.797287 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c04163-3c8f-4ace-906a-51015fa5d44b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:02.798760 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c04163-3c8f-4ace-906a-51015fa5d44b None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:02.815593 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-55c04163-3c8f-4ace-906a-51015fa5d44b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:24:02.815593 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:02.815593 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:24:02.815593 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:24:02.815593 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:24:02.815593 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:24:02.815593 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:24:02.815593 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:24:02.815593 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:24:02.815593 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:24:02.815593 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:24:02.815593 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:24:02.815593 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 08:24:02.815593 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 08:24:02.815593 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 08:24:02.815593 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 08:24:02.816637 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:24:02.816637 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:24:02.816637 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:24:02.816637 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:24:02.816637 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:24:02.816637 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:24:02.816637 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:24:02.816637 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:24:02.816637 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:24:02.816637 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:24:02.816637 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:24:02.816637 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:24:02.816637 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:24:02.816637 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:24:02.816637 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:24:02.816637 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:24:02.816637 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 08:24:02.818016 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 08:24:02.818016 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:24:02.818016 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:24:02.818016 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 377/765] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:24:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:24:02.826121 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b528225-4b90-4be0-b1de-23a882b11902 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:02.969605 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b528225-4b90-4be0-b1de-23a882b11902 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:02.971144 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:03.020947 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be63d8b6-8de5-470e-aae8-0d9e550ab82e None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:03.021275 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be63d8b6-8de5-470e-aae8-0d9e550ab82e None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:03.021500 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be63d8b6-8de5-470e-aae8-0d9e550ab82e None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:03.023054 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 378/766] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:24:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:24:03.025719 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f290c02-5fa4-4846-91b9-c82fbe1c2906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:03.026349 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f290c02-5fa4-4846-91b9-c82fbe1c2906 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:03.026630 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f290c02-5fa4-4846-91b9-c82fbe1c2906 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:03.026898 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f290c02-5fa4-4846-91b9-c82fbe1c2906 None admin] PATH_INFO: `/v3/domains` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:03.027562 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f290c02-5fa4-4846-91b9-c82fbe1c2906 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:03.027865 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f290c02-5fa4-4846-91b9-c82fbe1c2906 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:03.029305 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f290c02-5fa4-4846-91b9-c82fbe1c2906 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:03.030144 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf03734-7e33-4930-9433-f4bff12bb10c None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:03.030430 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf03734-7e33-4930-9433-f4bff12bb10c None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:03.030730 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf03734-7e33-4930-9433-f4bff12bb10c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:03.043927 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 389/767] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Aug 10 08:24:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:03.049226 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f290c02-5fa4-4846-91b9-c82fbe1c2906 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:03.066872 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-fbf03734-7e33-4930-9433-f4bff12bb10c None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:24:03.068079 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-fbf03734-7e33-4930-9433-f4bff12bb10c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:24:03.099762 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:03.179670 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f290c02-5fa4-4846-91b9-c82fbe1c2906 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:03.188029 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 379/768] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:24:03 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 08:24:03.212093 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf03734-7e33-4930-9433-f4bff12bb10c None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:03.221045 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cae43e6-d8c5-4970-bd3c-a7b7c5fb3170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:03.221722 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cae43e6-d8c5-4970-bd3c-a7b7c5fb3170 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:03.222078 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cae43e6-d8c5-4970-bd3c-a7b7c5fb3170 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:03.222361 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cae43e6-d8c5-4970-bd3c-a7b7c5fb3170 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:03.230484 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cae43e6-d8c5-4970-bd3c-a7b7c5fb3170 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:03.231943 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cae43e6-d8c5-4970-bd3c-a7b7c5fb3170 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:03.244035 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-2cae43e6-d8c5-4970-bd3c-a7b7c5fb3170 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 08:24:03.244035 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:03.244035 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:24:03.244035 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:24:03.244035 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:24:03.244035 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:24:03.244035 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:24:03.244035 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:24:03.244035 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:24:03.244035 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:24:03.244035 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:24:03.244035 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:24:03.244035 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 08:24:03.244035 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 08:24:03.244035 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 08:24:03.244035 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 08:24:03.245347 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:24:03.245347 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:24:03.245347 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 08:24:03.245347 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 08:24:03.245347 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 08:24:03.245347 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 08:24:03.245347 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:24:03.245347 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:24:03.245347 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:24:03.245347 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:24:03.245347 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:24:03.245347 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:24:03.245347 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:24:03.245347 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:24:03.245347 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:24:03.245347 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:24:03.245347 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:24:03.245347 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:24:03.246938 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:24:03.246938 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:24:03.246938 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 08:24:03.246938 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 08:24:03.246938 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 08:24:03.246938 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 08:24:03.246938 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 08:24:03.246938 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 08:24:03.246938 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 08:24:03.246938 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:24:03.246938 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 390/769] 10.222.0.57 () {56 vars in 1093 bytes} [Wed Aug 10 08:24:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:24:03.251369 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cae43e6-d8c5-4970-bd3c-a7b7c5fb3170 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:03.277941 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:03.357187 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cae43e6-d8c5-4970-bd3c-a7b7c5fb3170 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:03.380478 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf03734-7e33-4930-9433-f4bff12bb10c None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:03.401909 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5a9e75-6ff3-47d9-8b8b-a99415585eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:03.402255 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5a9e75-6ff3-47d9-8b8b-a99415585eb0 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:03.402378 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5a9e75-6ff3-47d9-8b8b-a99415585eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:03.404940 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5a9e75-6ff3-47d9-8b8b-a99415585eb0 None admin] PATH_INFO: `/v3/users` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:03.404940 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5a9e75-6ff3-47d9-8b8b-a99415585eb0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:03.404940 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5a9e75-6ff3-47d9-8b8b-a99415585eb0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:03.405837 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5a9e75-6ff3-47d9-8b8b-a99415585eb0 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:03.436222 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 391/770] 10.222.0.57 () {56 vars in 1133 bytes} [Wed Aug 10 08:24:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:24:03.441886 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5a9e75-6ff3-47d9-8b8b-a99415585eb0 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:03.463010 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f232c482-35e7-4e17-b5d2-862ff48ffebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:03.463672 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f232c482-35e7-4e17-b5d2-862ff48ffebe None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:03.463913 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f232c482-35e7-4e17-b5d2-862ff48ffebe None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:03.464181 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f232c482-35e7-4e17-b5d2-862ff48ffebe None admin] PATH_INFO: `/v3/services/network` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:03.464836 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f232c482-35e7-4e17-b5d2-862ff48ffebe None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:03.466246 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f232c482-35e7-4e17-b5d2-862ff48ffebe None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:03.481652 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-f232c482-35e7-4e17-b5d2-862ff48ffebe None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 08:24:03.481652 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:03.481652 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 08:24:03.481652 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 08:24:03.481652 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 08:24:03.481652 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 08:24:03.481652 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 08:24:03.481652 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 08:24:03.481652 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 08:24:03.481652 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:24:03.481652 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 08:24:03.481652 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:24:03.481652 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:03.481652 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:24:03.481652 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:24:03.481652 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:24:03.481652 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:24:03.484095 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:24:03.484095 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:24:03.484095 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:24:03.484095 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:24:03.484095 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:24:03.484095 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:24:03.484095 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 08:24:03.484095 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 08:24:03.484095 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 08:24:03.484095 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 08:24:03.484095 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:24:03.484095 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:24:03.484095 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:24:03.484095 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:24:03.484095 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:24:03.484095 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:24:03.486251 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:24:03.486251 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:24:03.486251 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:24:03.486251 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:24:03.486251 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:24:03.486251 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:24:03.486251 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:24:03.486251 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:24:03.486251 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:24:03.486251 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:24:03.486251 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 08:24:03.486251 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 08:24:03.486251 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 08:24:03.486251 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:24:03.486251 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 380/771] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Aug 10 08:24:03 2022] GET /identity/v3/services/network => generated 88 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 08:24:03.494053 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f232c482-35e7-4e17-b5d2-862ff48ffebe None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:03.518001 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:03.527919 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:03.548639 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5a9e75-6ff3-47d9-8b8b-a99415585eb0 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:03.588047 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ab1d98-ec46-4b95-b6d6-31c55065746f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:03.588686 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ab1d98-ec46-4b95-b6d6-31c55065746f None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:03.588995 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ab1d98-ec46-4b95-b6d6-31c55065746f None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:03.589272 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ab1d98-ec46-4b95-b6d6-31c55065746f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:03.599903 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ab1d98-ec46-4b95-b6d6-31c55065746f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:03.601410 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ab1d98-ec46-4b95-b6d6-31c55065746f None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:03.617116 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-38ab1d98-ec46-4b95-b6d6-31c55065746f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:24:03.617116 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:03.617116 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:24:03.617116 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:24:03.617116 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:24:03.617116 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:24:03.617116 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:24:03.617116 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:24:03.617116 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:24:03.617116 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:24:03.617116 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:24:03.617116 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:24:03.617116 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 08:24:03.617116 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 08:24:03.617116 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 08:24:03.617116 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 08:24:03.618122 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:24:03.618122 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:24:03.618122 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:24:03.618122 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:24:03.618122 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:24:03.618122 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:24:03.618122 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:24:03.618122 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:24:03.618122 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:24:03.618122 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:24:03.618122 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:24:03.618122 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:24:03.618122 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:24:03.618122 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:24:03.618122 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:24:03.618122 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:24:03.618122 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 08:24:03.619618 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 08:24:03.619618 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:24:03.619618 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:24:03.619618 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 392/772] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:24:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 08:24:03.624194 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ab1d98-ec46-4b95-b6d6-31c55065746f None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:03.669291 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f232c482-35e7-4e17-b5d2-862ff48ffebe None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:03.706178 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1164cc0c-95e2-4fe7-9790-51f24c009471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:03.706729 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1164cc0c-95e2-4fe7-9790-51f24c009471 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:03.707005 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1164cc0c-95e2-4fe7-9790-51f24c009471 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:03.707272 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1164cc0c-95e2-4fe7-9790-51f24c009471 None admin] PATH_INFO: `/v3/services` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:03.707930 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1164cc0c-95e2-4fe7-9790-51f24c009471 None admin] RBAC: Adding query filter params (name=network) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:03.708244 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1164cc0c-95e2-4fe7-9790-51f24c009471 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:03.709719 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1164cc0c-95e2-4fe7-9790-51f24c009471 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:03.714724 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 381/773] 10.222.0.57 () {56 vars in 1112 bytes} [Wed Aug 10 08:24:03 2022] GET /identity/v3/services?name=network => generated 124 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:24:03.715876 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1861a47-3c78-473f-9a66-47951b7649e7 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:03.716156 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1861a47-3c78-473f-9a66-47951b7649e7 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:03.716427 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1861a47-3c78-473f-9a66-47951b7649e7 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:03.722769 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 382/774] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:24:03 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:24:03.723544 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1861a47-3c78-473f-9a66-47951b7649e7 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:03.765795 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ab1d98-ec46-4b95-b6d6-31c55065746f None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:03.800379 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:03.806322 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1041e9be-3414-4830-9a41-8de1317d9b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:03.807066 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1041e9be-3414-4830-9a41-8de1317d9b4c None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:03.807622 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1041e9be-3414-4830-9a41-8de1317d9b4c None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:03.807901 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1041e9be-3414-4830-9a41-8de1317d9b4c None admin] PATH_INFO: `/v3/domains` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:03.808680 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1041e9be-3414-4830-9a41-8de1317d9b4c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:03.809183 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1041e9be-3414-4830-9a41-8de1317d9b4c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:03.813493 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1041e9be-3414-4830-9a41-8de1317d9b4c None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:03.816053 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:03.827078 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 393/775] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Aug 10 08:24:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:24:03.828345 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154246ad-6df5-482a-b891-43744fbc87ef None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:03.828629 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154246ad-6df5-482a-b891-43744fbc87ef None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:03.828905 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154246ad-6df5-482a-b891-43744fbc87ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:03.851811 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-154246ad-6df5-482a-b891-43744fbc87ef None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:24:03.852657 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1861a47-3c78-473f-9a66-47951b7649e7 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:03.853079 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-154246ad-6df5-482a-b891-43744fbc87ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:24:03.903464 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b640395-b79c-465f-ba1b-efc6d6303aa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:03.904241 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b640395-b79c-465f-ba1b-efc6d6303aa9 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:03.904592 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b640395-b79c-465f-ba1b-efc6d6303aa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:03.905436 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b640395-b79c-465f-ba1b-efc6d6303aa9 None admin] PATH_INFO: `/v3/services` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:03.906233 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b640395-b79c-465f-ba1b-efc6d6303aa9 None admin] RBAC: Adding query filter params (type=network) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:03.907015 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b640395-b79c-465f-ba1b-efc6d6303aa9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:03.908772 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b640395-b79c-465f-ba1b-efc6d6303aa9 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:03.916557 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 383/776] 10.222.0.57 () {56 vars in 1112 bytes} [Wed Aug 10 08:24:03 2022] GET /identity/v3/services?type=network => generated 350 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:03.917607 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b640395-b79c-465f-ba1b-efc6d6303aa9 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:03.938995 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 394/777] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:24:03 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 08:24:03.939855 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154246ad-6df5-482a-b891-43744fbc87ef None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:04.059793 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154246ad-6df5-482a-b891-43744fbc87ef None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:04.072414 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b640395-b79c-465f-ba1b-efc6d6303aa9 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:04.084974 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:04.099726 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb96d53-4e3e-418e-ab6c-49e86b398cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:04.100482 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb96d53-4e3e-418e-ab6c-49e86b398cf4 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:04.101002 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb96d53-4e3e-418e-ab6c-49e86b398cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:04.101375 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb96d53-4e3e-418e-ab6c-49e86b398cf4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:04.102185 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb96d53-4e3e-418e-ab6c-49e86b398cf4 None admin] RBAC: Adding query filter params (interface=public, service_id=837b14b18fe54052b8bea812edc60988, region_id=RegionOne) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:04.102601 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb96d53-4e3e-418e-ab6c-49e86b398cf4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:04.104510 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb96d53-4e3e-418e-ab6c-49e86b398cf4 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:04.115585 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 395/778] 10.222.0.57 () {56 vars in 1251 bytes} [Wed Aug 10 08:24:03 2022] GET /identity/v3/endpoints?service_id=837b14b18fe54052b8bea812edc60988&interface=public®ion_id=RegionOne => generated 194 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:04.116715 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb96d53-4e3e-418e-ab6c-49e86b398cf4 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:04.117884 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3914c68-74dc-4a55-8f94-36f65c19959c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:04.118530 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3914c68-74dc-4a55-8f94-36f65c19959c None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:04.118808 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3914c68-74dc-4a55-8f94-36f65c19959c None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:04.119076 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3914c68-74dc-4a55-8f94-36f65c19959c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:04.128645 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3914c68-74dc-4a55-8f94-36f65c19959c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:04.130643 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3914c68-74dc-4a55-8f94-36f65c19959c None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:04.133312 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:04.146962 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-d3914c68-74dc-4a55-8f94-36f65c19959c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 08:24:04.146962 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:04.146962 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:24:04.146962 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:24:04.146962 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:24:04.146962 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:24:04.146962 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:24:04.146962 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:24:04.146962 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:24:04.146962 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:24:04.146962 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:24:04.146962 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:24:04.146962 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 08:24:04.146962 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 08:24:04.146962 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 08:24:04.147959 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 08:24:04.147959 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:24:04.147959 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:24:04.147959 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:24:04.147959 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:24:04.147959 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:24:04.147959 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:24:04.147959 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:24:04.147959 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:24:04.147959 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:24:04.147959 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:24:04.147959 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:24:04.147959 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:24:04.147959 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:24:04.147959 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:24:04.147959 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:24:04.147959 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:24:04.148872 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 08:24:04.148872 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 08:24:04.148872 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 08:24:04.148872 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 08:24:04.148872 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 08:24:04.148872 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 08:24:04.148872 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 08:24:04.148872 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:24:04.148872 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 384/779] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Aug 10 08:24:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:24:04.149415 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3914c68-74dc-4a55-8f94-36f65c19959c None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:04.263004 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb96d53-4e3e-418e-ab6c-49e86b398cf4 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:04.269994 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3914c68-74dc-4a55-8f94-36f65c19959c None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:04.303947 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e97e6e1-2641-4953-8c32-be2a9f945492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:04.304995 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e97e6e1-2641-4953-8c32-be2a9f945492 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:04.305491 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e97e6e1-2641-4953-8c32-be2a9f945492 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:04.306069 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e97e6e1-2641-4953-8c32-be2a9f945492 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:04.316884 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfdc798-8603-40ce-b037-96e3831fe0a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:04.317462 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfdc798-8603-40ce-b037-96e3831fe0a2 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:04.317744 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfdc798-8603-40ce-b037-96e3831fe0a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:04.318016 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfdc798-8603-40ce-b037-96e3831fe0a2 None admin] PATH_INFO: `/v3/services` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:04.318920 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfdc798-8603-40ce-b037-96e3831fe0a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:04.319450 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e97e6e1-2641-4953-8c32-be2a9f945492 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 08:24:04.320295 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfdc798-8603-40ce-b037-96e3831fe0a2 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:04.324939 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-6e97e6e1-2641-4953-8c32-be2a9f945492 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 08:24:04.324939 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:04.324939 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:24:04.324939 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:24:04.324939 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:24:04.324939 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:24:04.324939 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:24:04.324939 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:24:04.324939 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:24:04.324939 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:24:04.324939 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:24:04.324939 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:24:04.324939 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 08:24:04.324939 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 08:24:04.324939 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 08:24:04.324939 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 08:24:04.325963 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 08:24:04.325963 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 08:24:04.325963 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 08:24:04.325963 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 08:24:04.325963 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:24:04.325963 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:24:04.325963 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:24:04.325963 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:24:04.325963 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:24:04.325963 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:24:04.325963 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:24:04.325963 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:24:04.325963 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:24:04.325963 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:24:04.325963 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:24:04.325963 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:24:04.326835 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:24:04.326835 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:24:04.326835 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:24:04.326835 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:24:04.326835 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 08:24:04.326835 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 08:24:04.326835 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 08:24:04.326835 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 08:24:04.326835 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 08:24:04.326835 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 08:24:04.326835 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 08:24:04.326835 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:24:04.329753 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 396/781] 10.222.0.57 () {56 vars in 1102 bytes} [Wed Aug 10 08:24:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:24:04.329881 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 385/781] 10.222.0.57 () {56 vars in 1087 bytes} [Wed Aug 10 08:24:04 2022] GET /identity/v3/services => generated 772 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:04.330830 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfdc798-8603-40ce-b037-96e3831fe0a2 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:04.342310 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e97e6e1-2641-4953-8c32-be2a9f945492 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:04.450821 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:04.467849 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:04.511688 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfdc798-8603-40ce-b037-96e3831fe0a2 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:04.555335 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6a0223-fbcf-4ebc-b900-83fd847dbb39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:04.555973 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6a0223-fbcf-4ebc-b900-83fd847dbb39 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:04.556254 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6a0223-fbcf-4ebc-b900-83fd847dbb39 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:04.556383 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e97e6e1-2641-4953-8c32-be2a9f945492 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:04.556551 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6a0223-fbcf-4ebc-b900-83fd847dbb39 None admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:04.558188 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6a0223-fbcf-4ebc-b900-83fd847dbb39 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:04.558188 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6a0223-fbcf-4ebc-b900-83fd847dbb39 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:04.560563 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6a0223-fbcf-4ebc-b900-83fd847dbb39 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:04.575060 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 386/782] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Aug 10 08:24:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:04.577053 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1346a1-68e1-4166-a083-6edfd19f3661 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:04.577337 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1346a1-68e1-4166-a083-6edfd19f3661 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:04.577634 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1346a1-68e1-4166-a083-6edfd19f3661 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:04.579098 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 387/783] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:24:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:24:04.584955 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7537ee17-1ea4-4ebc-9031-7d904340f8de None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:04.585215 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7537ee17-1ea4-4ebc-9031-7d904340f8de None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:04.585414 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7537ee17-1ea4-4ebc-9031-7d904340f8de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:04.600284 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa58315a-30b4-465b-a5d8-940dcd4d7f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:04.600895 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa58315a-30b4-465b-a5d8-940dcd4d7f8d None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:04.601211 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa58315a-30b4-465b-a5d8-940dcd4d7f8d None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:04.601486 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa58315a-30b4-465b-a5d8-940dcd4d7f8d None admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:04.602149 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa58315a-30b4-465b-a5d8-940dcd4d7f8d None admin] RBAC: Adding query filter params (name=service) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:04.602454 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa58315a-30b4-465b-a5d8-940dcd4d7f8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:04.603830 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa58315a-30b4-465b-a5d8-940dcd4d7f8d None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:04.608777 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-7537ee17-1ea4-4ebc-9031-7d904340f8de None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:24:04.610174 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-7537ee17-1ea4-4ebc-9031-7d904340f8de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:24:04.612876 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 397/784] 10.222.0.57 () {56 vars in 1103 bytes} [Wed Aug 10 08:24:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:24:04.613913 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa58315a-30b4-465b-a5d8-940dcd4d7f8d None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:04.680644 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 388/785] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:24:04 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 08:24:04.681560 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7537ee17-1ea4-4ebc-9031-7d904340f8de None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:04.746425 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa58315a-30b4-465b-a5d8-940dcd4d7f8d None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:04.747312 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:04.757336 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:04.781567 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f2ff38-a4ac-44ad-a2cb-497eb6dd2f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:04.782231 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f2ff38-a4ac-44ad-a2cb-497eb6dd2f0b None admin] REQUEST_METHOD: `PUT` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:04.782514 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f2ff38-a4ac-44ad-a2cb-497eb6dd2f0b None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:04.782789 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f2ff38-a4ac-44ad-a2cb-497eb6dd2f0b None admin] PATH_INFO: `/v3/projects/b49e381419e940278bcb6111899589b5/users/a943008193914f76a663d489bd73e873/roles/ec3751692d9c4ecf81b4a71c936c46be` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:04.785647 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f2ff38-a4ac-44ad-a2cb-497eb6dd2f0b None admin] RBAC: Authorizing `identity:create_grant(project_id=b49e381419e940278bcb6111899589b5, user_id=a943008193914f76a663d489bd73e873, role_id=ec3751692d9c4ecf81b4a71c936c46be)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:04.787102 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f2ff38-a4ac-44ad-a2cb-497eb6dd2f0b None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:04.797034 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7537ee17-1ea4-4ebc-9031-7d904340f8de None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:04.832731 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47203e21-9c32-4572-848a-08af93195de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:04.833427 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47203e21-9c32-4572-848a-08af93195de5 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:04.833705 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47203e21-9c32-4572-848a-08af93195de5 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:04.833972 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47203e21-9c32-4572-848a-08af93195de5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:04.844499 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47203e21-9c32-4572-848a-08af93195de5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:04.846270 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47203e21-9c32-4572-848a-08af93195de5 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:04.861053 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 398/786] 10.222.0.57 () {58 vars in 1439 bytes} [Wed Aug 10 08:24:04 2022] PUT /identity/v3/projects/b49e381419e940278bcb6111899589b5/users/a943008193914f76a663d489bd73e873/roles/ec3751692d9c4ecf81b4a71c936c46be => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:24:04.864025 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f2ff38-a4ac-44ad-a2cb-497eb6dd2f0b None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:04.873925 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-47203e21-9c32-4572-848a-08af93195de5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:24:04.873925 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:04.873925 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:24:04.873925 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:24:04.873925 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:24:04.873925 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:24:04.873925 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:24:04.873925 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:24:04.873925 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:24:04.873925 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:24:04.873925 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:24:04.873925 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:24:04.873925 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 08:24:04.873925 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 08:24:04.873925 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 08:24:04.873925 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 08:24:04.874983 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:24:04.874983 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:24:04.874983 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:24:04.874983 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:24:04.874983 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:24:04.874983 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:24:04.874983 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:24:04.874983 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:24:04.874983 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:24:04.874983 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:24:04.874983 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:24:04.874983 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:24:04.874983 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:24:04.874983 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:24:04.874983 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:24:04.874983 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:24:04.874983 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 08:24:04.875904 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 08:24:04.875904 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:24:04.875904 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:24:04.879306 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 389/787] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:24:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:24:04.883572 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47203e21-9c32-4572-848a-08af93195de5 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:05.047348 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f2ff38-a4ac-44ad-a2cb-497eb6dd2f0b None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:05.055929 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47203e21-9c32-4572-848a-08af93195de5 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:05.058389 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:05.086160 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23897155-c940-4892-bd00-a3519ebc37a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:05.086926 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23897155-c940-4892-bd00-a3519ebc37a9 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:05.087245 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23897155-c940-4892-bd00-a3519ebc37a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:05.087631 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23897155-c940-4892-bd00-a3519ebc37a9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:05.094004 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d8c102-aa4c-4b27-9f83-2dcbd59eaffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:05.094839 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d8c102-aa4c-4b27-9f83-2dcbd59eaffb None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:05.095299 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d8c102-aa4c-4b27-9f83-2dcbd59eaffb None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:05.095748 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d8c102-aa4c-4b27-9f83-2dcbd59eaffb None admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:05.097164 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d8c102-aa4c-4b27-9f83-2dcbd59eaffb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:05.097676 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d8c102-aa4c-4b27-9f83-2dcbd59eaffb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:05.099226 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d8c102-aa4c-4b27-9f83-2dcbd59eaffb None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:05.099963 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23897155-c940-4892-bd00-a3519ebc37a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 08:24:05.106102 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-23897155-c940-4892-bd00-a3519ebc37a9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 08:24:05.106102 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:05.106102 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:24:05.106102 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:24:05.106102 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:24:05.106102 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:24:05.106102 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:24:05.106102 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:24:05.106102 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:24:05.106102 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:24:05.106102 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:24:05.106102 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:24:05.106102 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 08:24:05.106102 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 08:24:05.106102 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 08:24:05.106102 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 08:24:05.107088 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 08:24:05.107088 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 08:24:05.107088 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 08:24:05.107088 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 08:24:05.107088 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:24:05.107088 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:24:05.107088 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:24:05.107088 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:24:05.107088 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:24:05.107088 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:24:05.107088 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:24:05.107088 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:24:05.107088 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:24:05.107088 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:24:05.107088 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:24:05.107088 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:24:05.107952 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:24:05.107952 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:24:05.107952 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:24:05.107952 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:24:05.107952 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 08:24:05.107952 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 08:24:05.107952 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 08:24:05.107952 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 08:24:05.107952 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 08:24:05.107952 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 08:24:05.107952 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 08:24:05.107952 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:24:05.107952 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 399/788] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Aug 10 08:24:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:24:05.114785 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 390/789] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Aug 10 08:24:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:05.116701 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23897155-c940-4892-bd00-a3519ebc37a9 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:05.126995 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d8c102-aa4c-4b27-9f83-2dcbd59eaffb None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:05.151949 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:05.235505 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23897155-c940-4892-bd00-a3519ebc37a9 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:05.252317 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d8c102-aa4c-4b27-9f83-2dcbd59eaffb None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:05.274357 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd080cc1-398f-47fd-a753-343a6eb739b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:05.275381 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd080cc1-398f-47fd-a753-343a6eb739b7 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:05.275889 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd080cc1-398f-47fd-a753-343a6eb739b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:05.276378 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd080cc1-398f-47fd-a753-343a6eb739b7 None admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:05.277297 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd080cc1-398f-47fd-a753-343a6eb739b7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:05.277881 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd080cc1-398f-47fd-a753-343a6eb739b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:05.279447 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd080cc1-398f-47fd-a753-343a6eb739b7 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:05.289026 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 400/790] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Aug 10 08:24:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:05.294791 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd080cc1-398f-47fd-a753-343a6eb739b7 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:05.302040 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95591e6a-775e-40ac-8288-3dbc11154762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:05.304607 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95591e6a-775e-40ac-8288-3dbc11154762 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:05.305329 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95591e6a-775e-40ac-8288-3dbc11154762 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:05.306438 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95591e6a-775e-40ac-8288-3dbc11154762 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:05.323174 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95591e6a-775e-40ac-8288-3dbc11154762 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:05.324964 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95591e6a-775e-40ac-8288-3dbc11154762 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:05.350855 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-95591e6a-775e-40ac-8288-3dbc11154762 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 08:24:05.350855 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:05.350855 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:24:05.350855 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:24:05.350855 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:24:05.350855 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:24:05.350855 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:24:05.350855 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:24:05.350855 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:24:05.350855 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:24:05.350855 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:24:05.350855 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:24:05.350855 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 08:24:05.350855 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 08:24:05.350855 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 08:24:05.350855 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 08:24:05.351862 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:24:05.351862 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:24:05.351862 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 08:24:05.351862 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 08:24:05.351862 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 08:24:05.351862 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 08:24:05.351862 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:24:05.351862 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:24:05.351862 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:24:05.351862 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:24:05.351862 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:24:05.351862 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:24:05.351862 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:24:05.351862 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:24:05.351862 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:24:05.351862 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:24:05.351862 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:24:05.351862 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:24:05.352821 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:24:05.352821 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:24:05.352821 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 08:24:05.352821 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 08:24:05.352821 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 08:24:05.352821 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 08:24:05.352821 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 08:24:05.352821 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 08:24:05.352821 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 08:24:05.352821 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:24:05.358685 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 391/791] 10.222.0.57 () {56 vars in 1117 bytes} [Wed Aug 10 08:24:05 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:24:05.364804 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95591e6a-775e-40ac-8288-3dbc11154762 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:05.420637 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:05.428479 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:05.438479 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd080cc1-398f-47fd-a753-343a6eb739b7 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:05.473311 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a75e353-9c2a-4acf-a09a-569688d5af86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:05.475046 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a75e353-9c2a-4acf-a09a-569688d5af86 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:05.475046 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a75e353-9c2a-4acf-a09a-569688d5af86 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:05.475046 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a75e353-9c2a-4acf-a09a-569688d5af86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:05.486063 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a75e353-9c2a-4acf-a09a-569688d5af86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:05.488509 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a75e353-9c2a-4acf-a09a-569688d5af86 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:05.498724 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95591e6a-775e-40ac-8288-3dbc11154762 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:05.505807 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-7a75e353-9c2a-4acf-a09a-569688d5af86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:24:05.505807 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:05.505807 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:24:05.505807 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:24:05.505807 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:24:05.505807 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:24:05.505807 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:24:05.505807 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:24:05.505807 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:24:05.505807 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:24:05.505807 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:24:05.505807 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:24:05.505807 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 08:24:05.505807 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 08:24:05.505807 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 08:24:05.505807 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 08:24:05.506761 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:24:05.506761 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:24:05.506761 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:24:05.506761 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:24:05.506761 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:24:05.506761 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:24:05.506761 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:24:05.506761 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:24:05.506761 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:24:05.506761 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:24:05.506761 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:24:05.506761 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:24:05.506761 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:24:05.506761 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:24:05.506761 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:24:05.506761 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:24:05.506761 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 08:24:05.507669 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 08:24:05.507669 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:24:05.507669 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:24:05.514116 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 401/792] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:24:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 08:24:05.515664 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83edf083-06ef-42db-b99b-57f401d80c6b None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:05.516165 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83edf083-06ef-42db-b99b-57f401d80c6b None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:05.516637 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83edf083-06ef-42db-b99b-57f401d80c6b None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:05.518430 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 402/793] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:24:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:24:05.519545 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83edf083-06ef-42db-b99b-57f401d80c6b None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:05.537771 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9765d17-b3e3-4440-a240-ef5c847ba419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:05.538172 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9765d17-b3e3-4440-a240-ef5c847ba419 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:05.538447 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9765d17-b3e3-4440-a240-ef5c847ba419 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:05.538712 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9765d17-b3e3-4440-a240-ef5c847ba419 None admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:05.539447 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9765d17-b3e3-4440-a240-ef5c847ba419 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:05.539747 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9765d17-b3e3-4440-a240-ef5c847ba419 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:05.541276 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9765d17-b3e3-4440-a240-ef5c847ba419 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:05.560404 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 392/794] 10.222.0.57 () {56 vars in 1149 bytes} [Wed Aug 10 08:24:05 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:24:05.561694 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73916e76-6d76-4fba-acf4-b18a69ad9244 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:05.561974 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73916e76-6d76-4fba-acf4-b18a69ad9244 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:05.562243 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73916e76-6d76-4fba-acf4-b18a69ad9244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:05.579980 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-73916e76-6d76-4fba-acf4-b18a69ad9244 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:24:05.581306 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-73916e76-6d76-4fba-acf4-b18a69ad9244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:24:05.633399 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83edf083-06ef-42db-b99b-57f401d80c6b None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:05.655438 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 393/795] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:24:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 08:24:05.656711 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73916e76-6d76-4fba-acf4-b18a69ad9244 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:05.671938 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553158b1-1b6f-44f9-bcad-a3de608437dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:05.672896 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553158b1-1b6f-44f9-bcad-a3de608437dd None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:05.673419 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553158b1-1b6f-44f9-bcad-a3de608437dd None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:05.673890 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553158b1-1b6f-44f9-bcad-a3de608437dd None admin] PATH_INFO: `/v3/domains` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:05.674817 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553158b1-1b6f-44f9-bcad-a3de608437dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:05.675321 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553158b1-1b6f-44f9-bcad-a3de608437dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:05.676998 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553158b1-1b6f-44f9-bcad-a3de608437dd None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:05.681493 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:05.690722 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 403/796] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Aug 10 08:24:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:05.698702 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553158b1-1b6f-44f9-bcad-a3de608437dd None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:05.704004 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:05.789817 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73916e76-6d76-4fba-acf4-b18a69ad9244 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:05.814197 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553158b1-1b6f-44f9-bcad-a3de608437dd None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:05.835959 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e346102-b9f2-4650-9bb1-a318e80e5263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:05.836582 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e346102-b9f2-4650-9bb1-a318e80e5263 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:05.836855 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e346102-b9f2-4650-9bb1-a318e80e5263 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:05.837177 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e346102-b9f2-4650-9bb1-a318e80e5263 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:05.850403 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e346102-b9f2-4650-9bb1-a318e80e5263 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:05.851835 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e346102-b9f2-4650-9bb1-a318e80e5263 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:05.853514 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ea1095-5d5a-4565-b672-4ea1b7e8f665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:05.854398 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ea1095-5d5a-4565-b672-4ea1b7e8f665 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:05.854886 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ea1095-5d5a-4565-b672-4ea1b7e8f665 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:05.855355 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ea1095-5d5a-4565-b672-4ea1b7e8f665 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:05.863254 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ea1095-5d5a-4565-b672-4ea1b7e8f665 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:05.864950 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ea1095-5d5a-4565-b672-4ea1b7e8f665 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:05.868347 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-4e346102-b9f2-4650-9bb1-a318e80e5263 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:24:05.868347 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:05.868347 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:24:05.868347 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:24:05.868347 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:24:05.868347 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:24:05.868347 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:24:05.868347 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:24:05.868347 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:24:05.868347 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:24:05.868347 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:24:05.868347 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:24:05.868347 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 08:24:05.868347 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 08:24:05.868347 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 08:24:05.868347 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 08:24:05.869335 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:24:05.869335 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:24:05.869335 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:24:05.869335 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:24:05.869335 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:24:05.869335 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:24:05.869335 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:24:05.869335 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:24:05.869335 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:24:05.869335 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:24:05.869335 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:24:05.869335 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:24:05.869335 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:24:05.869335 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:24:05.869335 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:24:05.869335 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:24:05.869335 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 08:24:05.870517 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 08:24:05.870517 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:24:05.870517 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:24:05.870517 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 394/797] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:24:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:24:05.871262 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e346102-b9f2-4650-9bb1-a318e80e5263 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:05.881472 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-43ea1095-5d5a-4565-b672-4ea1b7e8f665 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 10 08:24:05.881472 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:05.881472 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:24:05.881472 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:24:05.881472 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:24:05.881472 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:24:05.881472 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:24:05.881472 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:24:05.881472 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:24:05.881472 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:24:05.881472 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:24:05.881472 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:24:05.881472 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 08:24:05.881472 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 08:24:05.881472 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 08:24:05.881472 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 08:24:05.882434 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:24:05.882434 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:24:05.882434 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 08:24:05.882434 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 08:24:05.882434 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 08:24:05.882434 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 08:24:05.882434 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:24:05.882434 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:24:05.882434 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:24:05.882434 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:24:05.882434 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:24:05.882434 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:24:05.882434 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:24:05.882434 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:24:05.882434 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:24:05.882434 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:24:05.882434 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:24:05.882434 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:24:05.883390 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:24:05.883390 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:24:05.883390 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 08:24:05.883390 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 08:24:05.883390 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 08:24:05.883390 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 08:24:05.883390 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 08:24:05.883390 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 08:24:05.883390 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 10 08:24:05.883390 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:24:05.890456 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 404/798] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Aug 10 08:24:05 2022] GET /identity/v3/users/swift => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 08:24:05.891529 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ea1095-5d5a-4565-b672-4ea1b7e8f665 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:05.951908 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:05.956446 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:06.010381 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e346102-b9f2-4650-9bb1-a318e80e5263 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:06.010861 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ea1095-5d5a-4565-b672-4ea1b7e8f665 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:06.046489 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c9f2d4-fece-4521-92d4-f46514d94a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:06.047665 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c9f2d4-fece-4521-92d4-f46514d94a7f None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:06.048149 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c9f2d4-fece-4521-92d4-f46514d94a7f None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:06.048637 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c9f2d4-fece-4521-92d4-f46514d94a7f None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:06.050002 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c9f2d4-fece-4521-92d4-f46514d94a7f None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:06.051944 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c9f2d4-fece-4521-92d4-f46514d94a7f None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:06.055952 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2864424e-4ee6-4bd8-8a3e-f1a3d31a7564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:06.056563 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2864424e-4ee6-4bd8-8a3e-f1a3d31a7564 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:06.056842 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2864424e-4ee6-4bd8-8a3e-f1a3d31a7564 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:06.057287 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2864424e-4ee6-4bd8-8a3e-f1a3d31a7564 None admin] PATH_INFO: `/v3/domains` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:06.058035 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2864424e-4ee6-4bd8-8a3e-f1a3d31a7564 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:06.058339 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2864424e-4ee6-4bd8-8a3e-f1a3d31a7564 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:06.059692 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2864424e-4ee6-4bd8-8a3e-f1a3d31a7564 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:06.063419 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-12c9f2d4-fece-4521-92d4-f46514d94a7f None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 08:24:06.063419 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:06.063419 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 08:24:06.063419 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 08:24:06.063419 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 08:24:06.063419 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 08:24:06.063419 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 08:24:06.063419 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 08:24:06.063419 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 08:24:06.063419 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:24:06.063419 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 08:24:06.063419 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:24:06.063419 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:06.063419 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:24:06.063419 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:24:06.063419 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:24:06.063419 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:24:06.064402 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:24:06.064402 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:24:06.064402 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:24:06.064402 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:24:06.064402 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:24:06.064402 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:24:06.064402 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 08:24:06.064402 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 08:24:06.064402 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 08:24:06.064402 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 08:24:06.064402 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:24:06.064402 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:24:06.064402 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:24:06.064402 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:24:06.064402 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:24:06.064402 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:24:06.065952 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:24:06.065952 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:24:06.065952 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:24:06.065952 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:24:06.065952 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:24:06.065952 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:24:06.065952 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:24:06.065952 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:24:06.065952 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:24:06.065952 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:24:06.065952 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 08:24:06.065952 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 08:24:06.065952 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 08:24:06.065952 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:24:06.068779 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 395/799] 10.222.0.57 () {56 vars in 1129 bytes} [Wed Aug 10 08:24:05 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:24:06.069809 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c9f2d4-fece-4521-92d4-f46514d94a7f None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:06.077960 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 405/800] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Aug 10 08:24:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:06.080538 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2864424e-4ee6-4bd8-8a3e-f1a3d31a7564 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:06.191782 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c9f2d4-fece-4521-92d4-f46514d94a7f None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:06.217960 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2864424e-4ee6-4bd8-8a3e-f1a3d31a7564 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:06.227223 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:06.238720 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d666e316-e0ea-46ee-849c-8721d73d1eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:06.239336 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d666e316-e0ea-46ee-849c-8721d73d1eb5 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:06.239614 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d666e316-e0ea-46ee-849c-8721d73d1eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:06.241765 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d666e316-e0ea-46ee-849c-8721d73d1eb5 None admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:06.241765 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d666e316-e0ea-46ee-849c-8721d73d1eb5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:06.241765 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d666e316-e0ea-46ee-849c-8721d73d1eb5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:06.243033 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d666e316-e0ea-46ee-849c-8721d73d1eb5 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:06.245994 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:06.259146 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7828872a-56db-45c8-bc43-f5fd3c6c67e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:06.260087 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7828872a-56db-45c8-bc43-f5fd3c6c67e7 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:06.260567 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7828872a-56db-45c8-bc43-f5fd3c6c67e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:06.261062 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7828872a-56db-45c8-bc43-f5fd3c6c67e7 None admin] PATH_INFO: `/v3/services` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:06.261893 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7828872a-56db-45c8-bc43-f5fd3c6c67e7 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:06.262408 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7828872a-56db-45c8-bc43-f5fd3c6c67e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:06.263970 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7828872a-56db-45c8-bc43-f5fd3c6c67e7 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:06.272444 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 396/801] 10.222.0.57 () {56 vars in 1135 bytes} [Wed Aug 10 08:24:06 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:06.272444 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d666e316-e0ea-46ee-849c-8721d73d1eb5 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:06.273722 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 406/802] 10.222.0.57 () {56 vars in 1124 bytes} [Wed Aug 10 08:24:06 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:24:06.274544 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7828872a-56db-45c8-bc43-f5fd3c6c67e7 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:06.381554 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d666e316-e0ea-46ee-849c-8721d73d1eb5 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:06.390592 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7828872a-56db-45c8-bc43-f5fd3c6c67e7 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:06.419732 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8c663f-74b3-4129-ac42-814ba363c497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:06.420692 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8c663f-74b3-4129-ac42-814ba363c497 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:06.421200 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8c663f-74b3-4129-ac42-814ba363c497 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:06.421698 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8c663f-74b3-4129-ac42-814ba363c497 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:06.429476 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8c663f-74b3-4129-ac42-814ba363c497 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:06.431550 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8c663f-74b3-4129-ac42-814ba363c497 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:06.432718 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cfe6b3-d4e1-4c1f-a1bc-6e94962040d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:06.433438 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cfe6b3-d4e1-4c1f-a1bc-6e94962040d9 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:06.433781 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cfe6b3-d4e1-4c1f-a1bc-6e94962040d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:06.434229 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cfe6b3-d4e1-4c1f-a1bc-6e94962040d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:06.444166 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-4b8c663f-74b3-4129-ac42-814ba363c497 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 08:24:06.444166 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:06.444166 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:24:06.444166 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:24:06.444166 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:24:06.444166 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:24:06.444166 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:24:06.444166 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:24:06.444166 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:24:06.444166 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:24:06.444166 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:24:06.444166 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:24:06.444166 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 08:24:06.444166 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 08:24:06.444166 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 08:24:06.445130 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 08:24:06.445130 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:24:06.445130 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:24:06.445130 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:24:06.445130 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:24:06.445130 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:24:06.445130 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:24:06.445130 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:24:06.445130 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:24:06.445130 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:24:06.445130 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:24:06.445130 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:24:06.445130 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:24:06.445130 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:24:06.445130 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:24:06.445130 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:24:06.445130 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:24:06.446064 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 08:24:06.446064 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 08:24:06.446064 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 08:24:06.446064 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 08:24:06.446064 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 08:24:06.446064 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 08:24:06.446064 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 08:24:06.446064 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:24:06.446516 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cfe6b3-d4e1-4c1f-a1bc-6e94962040d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:06.446516 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cfe6b3-d4e1-4c1f-a1bc-6e94962040d9 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:06.449028 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 397/803] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Aug 10 08:24:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:24:06.450022 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8c663f-74b3-4129-ac42-814ba363c497 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:06.462489 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-80cfe6b3-d4e1-4c1f-a1bc-6e94962040d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:24:06.462489 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:06.462489 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:24:06.462489 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:24:06.462489 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:24:06.462489 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:24:06.462489 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:24:06.462489 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:24:06.462489 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:24:06.462489 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:24:06.462489 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:24:06.462489 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:24:06.462489 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 08:24:06.462489 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 08:24:06.462489 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 08:24:06.462489 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 08:24:06.463508 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:24:06.463508 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:24:06.463508 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:24:06.463508 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:24:06.463508 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:24:06.463508 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:24:06.463508 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:24:06.463508 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:24:06.463508 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:24:06.463508 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:24:06.463508 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:24:06.463508 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:24:06.463508 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:24:06.463508 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:24:06.463508 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:24:06.463508 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:24:06.463508 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 08:24:06.464430 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 08:24:06.464430 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:24:06.464430 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:24:06.468410 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 407/804] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:24:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:24:06.469356 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cfe6b3-d4e1-4c1f-a1bc-6e94962040d9 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:06.503172 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:06.513109 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:06.560443 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8c663f-74b3-4129-ac42-814ba363c497 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:06.574654 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cfe6b3-d4e1-4c1f-a1bc-6e94962040d9 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:06.596024 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34ca12b-d84b-4c87-bf91-52937cbffad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:06.597567 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34ca12b-d84b-4c87-bf91-52937cbffad8 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:06.598051 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34ca12b-d84b-4c87-bf91-52937cbffad8 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:06.598573 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34ca12b-d84b-4c87-bf91-52937cbffad8 None admin] PATH_INFO: `/v3/services` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:06.599420 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34ca12b-d84b-4c87-bf91-52937cbffad8 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:06.599919 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34ca12b-d84b-4c87-bf91-52937cbffad8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:06.601547 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34ca12b-d84b-4c87-bf91-52937cbffad8 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:06.609227 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 398/805] 10.222.0.57 () {56 vars in 1124 bytes} [Wed Aug 10 08:24:06 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:06.610550 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34ca12b-d84b-4c87-bf91-52937cbffad8 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:06.614698 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f247f6cf-849f-4481-b81a-f29e9b7db38e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:06.615323 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f247f6cf-849f-4481-b81a-f29e9b7db38e None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:06.615599 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f247f6cf-849f-4481-b81a-f29e9b7db38e None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:06.615867 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f247f6cf-849f-4481-b81a-f29e9b7db38e None admin] PATH_INFO: `/v3/projects` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:06.616562 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f247f6cf-849f-4481-b81a-f29e9b7db38e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:06.616864 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f247f6cf-849f-4481-b81a-f29e9b7db38e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:06.618317 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f247f6cf-849f-4481-b81a-f29e9b7db38e None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:06.637712 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 408/806] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Aug 10 08:24:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:06.637712 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543c0eaa-0fac-4dca-9d75-ac278c45279e None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:06.637712 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543c0eaa-0fac-4dca-9d75-ac278c45279e None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:06.637712 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543c0eaa-0fac-4dca-9d75-ac278c45279e None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:06.637712 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 409/807] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:24:06 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:24:06.637712 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543c0eaa-0fac-4dca-9d75-ac278c45279e None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:06.725282 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34ca12b-d84b-4c87-bf91-52937cbffad8 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:06.735919 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:06.743434 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:06.748252 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543c0eaa-0fac-4dca-9d75-ac278c45279e None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:06.761627 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78318198-7bea-4789-88fa-aa4aa4cc7d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:06.762324 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78318198-7bea-4789-88fa-aa4aa4cc7d80 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:06.762764 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78318198-7bea-4789-88fa-aa4aa4cc7d80 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:06.763001 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78318198-7bea-4789-88fa-aa4aa4cc7d80 None admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:06.763728 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78318198-7bea-4789-88fa-aa4aa4cc7d80 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:06.764117 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78318198-7bea-4789-88fa-aa4aa4cc7d80 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:06.765822 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78318198-7bea-4789-88fa-aa4aa4cc7d80 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:06.782874 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 399/808] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Aug 10 08:24:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:06.782874 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78318198-7bea-4789-88fa-aa4aa4cc7d80 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:06.788980 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f89511-1f29-4d5a-b2c1-6246ab9d0034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:06.789944 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f89511-1f29-4d5a-b2c1-6246ab9d0034 None admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:06.790288 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f89511-1f29-4d5a-b2c1-6246ab9d0034 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:06.790562 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f89511-1f29-4d5a-b2c1-6246ab9d0034 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:06.791534 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f89511-1f29-4d5a-b2c1-6246ab9d0034 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:06.793207 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f89511-1f29-4d5a-b2c1-6246ab9d0034 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:06.876175 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 410/809] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 08:24:06 2022] POST /identity/v3/endpoints => generated 346 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:24:06.877248 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da061fd9-e906-457f-b504-ae9dbfb620f3 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:06.877592 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da061fd9-e906-457f-b504-ae9dbfb620f3 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:06.878522 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da061fd9-e906-457f-b504-ae9dbfb620f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:06.885253 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78318198-7bea-4789-88fa-aa4aa4cc7d80 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:06.901656 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-da061fd9-e906-457f-b504-ae9dbfb620f3 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:24:06.903289 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-da061fd9-e906-457f-b504-ae9dbfb620f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:24:06.937484 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2f6329-66f2-4f70-9ff7-7c585d45d009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:06.938238 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2f6329-66f2-4f70-9ff7-7c585d45d009 None admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:06.938523 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2f6329-66f2-4f70-9ff7-7c585d45d009 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:06.938797 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2f6329-66f2-4f70-9ff7-7c585d45d009 None admin] PATH_INFO: `/v3/projects/b49e381419e940278bcb6111899589b5/users/c140d31bd2b84c48a170e3ca90a041ce/roles/73fdd60e2f804ee69b76fa9145a472ff` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:06.957209 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2f6329-66f2-4f70-9ff7-7c585d45d009 None admin] RBAC: Authorizing `identity:create_grant(project_id=b49e381419e940278bcb6111899589b5, user_id=c140d31bd2b84c48a170e3ca90a041ce, role_id=73fdd60e2f804ee69b76fa9145a472ff)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:06.959069 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2f6329-66f2-4f70-9ff7-7c585d45d009 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:06.990469 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 411/810] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:24:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 08:24:06.991659 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da061fd9-e906-457f-b504-ae9dbfb620f3 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:07.033984 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 400/811] 10.222.0.57 () {58 vars in 1439 bytes} [Wed Aug 10 08:24:06 2022] PUT /identity/v3/projects/b49e381419e940278bcb6111899589b5/users/c140d31bd2b84c48a170e3ca90a041ce/roles/73fdd60e2f804ee69b76fa9145a472ff => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:24:07.035374 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2f6329-66f2-4f70-9ff7-7c585d45d009 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:07.063167 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:07.118674 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:07.141347 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da061fd9-e906-457f-b504-ae9dbfb620f3 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:07.191680 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2f6329-66f2-4f70-9ff7-7c585d45d009 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:07.198496 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1b5611-edb4-410f-837b-6a0a6c8f4ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:07.199567 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1b5611-edb4-410f-837b-6a0a6c8f4ef8 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:07.200054 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1b5611-edb4-410f-837b-6a0a6c8f4ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:07.201252 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1b5611-edb4-410f-837b-6a0a6c8f4ef8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:07.220371 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1b5611-edb4-410f-837b-6a0a6c8f4ef8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:07.222805 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1b5611-edb4-410f-837b-6a0a6c8f4ef8 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:07.236415 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-3b1b5611-edb4-410f-837b-6a0a6c8f4ef8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 08:24:07.236415 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:07.236415 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:24:07.236415 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:24:07.236415 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:24:07.236415 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:24:07.236415 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:24:07.236415 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:24:07.236415 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:24:07.236415 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:24:07.236415 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:24:07.236415 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:24:07.236415 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 08:24:07.236415 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 08:24:07.236415 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 08:24:07.237633 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 08:24:07.237633 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:24:07.237633 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:24:07.237633 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:24:07.237633 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:24:07.237633 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:24:07.237633 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:24:07.237633 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:24:07.237633 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:24:07.237633 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:24:07.237633 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:24:07.237633 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:24:07.237633 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:24:07.237633 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:24:07.237633 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:24:07.237633 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:24:07.237633 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:24:07.238565 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 08:24:07.238565 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 08:24:07.238565 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 08:24:07.238565 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 08:24:07.238565 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 08:24:07.238565 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 08:24:07.238565 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 08:24:07.238565 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:24:07.239038 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 412/812] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Aug 10 08:24:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:24:07.240166 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c61010-4914-42d6-8877-2812007267ea None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:07.240425 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c61010-4914-42d6-8877-2812007267ea None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:07.240752 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c61010-4914-42d6-8877-2812007267ea None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:07.241543 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0dd1ad-4981-4265-b648-e340ae7bd926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:07.242237 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 413/813] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:24:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:24:07.242688 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0dd1ad-4981-4265-b648-e340ae7bd926 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:07.242969 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0dd1ad-4981-4265-b648-e340ae7bd926 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:07.243238 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0dd1ad-4981-4265-b648-e340ae7bd926 None admin] PATH_INFO: `/v3/services/network` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:07.243863 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0dd1ad-4981-4265-b648-e340ae7bd926 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:07.246202 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0dd1ad-4981-4265-b648-e340ae7bd926 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:07.246315 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c61010-4914-42d6-8877-2812007267ea None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:07.255991 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-2a0dd1ad-4981-4265-b648-e340ae7bd926 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 08:24:07.255991 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:07.255991 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 08:24:07.255991 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 08:24:07.255991 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 08:24:07.255991 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 08:24:07.255991 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 08:24:07.255991 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 08:24:07.255991 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 08:24:07.255991 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:24:07.255991 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 08:24:07.255991 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:24:07.255991 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:07.255991 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:24:07.255991 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:24:07.255991 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:24:07.255991 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:24:07.257106 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:24:07.257106 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:24:07.257106 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:24:07.257106 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:24:07.257106 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:24:07.257106 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:24:07.257106 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 08:24:07.257106 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 08:24:07.257106 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 08:24:07.257106 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 08:24:07.257106 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:24:07.257106 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:24:07.257106 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:24:07.257106 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:24:07.257106 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:24:07.257106 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:24:07.258003 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:24:07.258003 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:24:07.258003 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:24:07.258003 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:24:07.258003 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:24:07.258003 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:24:07.258003 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:24:07.258003 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:24:07.258003 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:24:07.258003 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:24:07.258003 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 08:24:07.258003 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 08:24:07.258003 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 08:24:07.258003 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:24:07.258003 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 401/814] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Aug 10 08:24:07 2022] GET /identity/v3/services/network => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:24:07.258809 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b0de84-2d47-46d9-9112-570904e8a692 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:07.259112 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b0de84-2d47-46d9-9112-570904e8a692 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:07.259218 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b0de84-2d47-46d9-9112-570904e8a692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:07.280930 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-d6b0de84-2d47-46d9-9112-570904e8a692 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:24:07.282178 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-d6b0de84-2d47-46d9-9112-570904e8a692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:24:07.358805 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 402/815] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:24:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 08:24:07.359697 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b0de84-2d47-46d9-9112-570904e8a692 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:07.374130 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:07.379741 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c61010-4914-42d6-8877-2812007267ea None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:07.430842 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c663498-53c0-4c02-b6c8-9d9190520207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:07.432889 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c663498-53c0-4c02-b6c8-9d9190520207 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:07.433198 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c663498-53c0-4c02-b6c8-9d9190520207 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:07.433495 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c663498-53c0-4c02-b6c8-9d9190520207 None admin] PATH_INFO: `/v3/projects` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:07.434584 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c663498-53c0-4c02-b6c8-9d9190520207 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:07.434894 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c663498-53c0-4c02-b6c8-9d9190520207 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:07.436512 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c663498-53c0-4c02-b6c8-9d9190520207 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:07.455580 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 414/816] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Aug 10 08:24:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:07.457528 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c663498-53c0-4c02-b6c8-9d9190520207 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:07.463141 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:07.495587 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b0de84-2d47-46d9-9112-570904e8a692 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:07.535190 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89e2b79-b69b-4cad-b63c-a3cebbf779ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:07.535874 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89e2b79-b69b-4cad-b63c-a3cebbf779ca None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:07.536109 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89e2b79-b69b-4cad-b63c-a3cebbf779ca None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:07.536314 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89e2b79-b69b-4cad-b63c-a3cebbf779ca None admin] PATH_INFO: `/v3/services` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:07.536961 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89e2b79-b69b-4cad-b63c-a3cebbf779ca None admin] RBAC: Adding query filter params (name=network) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:07.537327 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89e2b79-b69b-4cad-b63c-a3cebbf779ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:07.538959 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89e2b79-b69b-4cad-b63c-a3cebbf779ca None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:07.548958 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 403/817] 10.222.0.57 () {56 vars in 1112 bytes} [Wed Aug 10 08:24:07 2022] GET /identity/v3/services?name=network => generated 124 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:07.548958 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89e2b79-b69b-4cad-b63c-a3cebbf779ca None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:07.602359 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c663498-53c0-4c02-b6c8-9d9190520207 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:07.641168 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf61149-2dd8-4268-8046-f123257173c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:07.641806 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf61149-2dd8-4268-8046-f123257173c8 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:07.642085 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf61149-2dd8-4268-8046-f123257173c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:07.642354 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf61149-2dd8-4268-8046-f123257173c8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:07.654496 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf61149-2dd8-4268-8046-f123257173c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 08:24:07.657979 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:07.662413 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-7bf61149-2dd8-4268-8046-f123257173c8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 08:24:07.662413 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:07.662413 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:24:07.662413 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:24:07.662413 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:24:07.662413 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:24:07.662413 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:24:07.662413 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:24:07.662413 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:24:07.662413 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:24:07.662413 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:24:07.662413 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:24:07.662413 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 08:24:07.662413 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 08:24:07.662413 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 08:24:07.662413 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 08:24:07.663518 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 08:24:07.663518 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 08:24:07.663518 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 08:24:07.663518 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 08:24:07.663518 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:24:07.663518 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:24:07.663518 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:24:07.663518 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:24:07.663518 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:24:07.663518 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:24:07.663518 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:24:07.663518 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:24:07.663518 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:24:07.663518 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:24:07.663518 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:24:07.663518 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:24:07.664566 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:24:07.664566 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:24:07.664566 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:24:07.664566 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:24:07.664566 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 08:24:07.664566 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 08:24:07.664566 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 08:24:07.664566 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 08:24:07.664566 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 08:24:07.664566 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 08:24:07.664566 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 08:24:07.664566 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:24:07.671616 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 415/818] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Aug 10 08:24:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:24:07.672521 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf61149-2dd8-4268-8046-f123257173c8 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:07.695897 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89e2b79-b69b-4cad-b63c-a3cebbf779ca None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:07.742651 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d4d9d9-fd3f-4319-9b67-3acabd58553a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:07.743586 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d4d9d9-fd3f-4319-9b67-3acabd58553a None admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:07.743957 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d4d9d9-fd3f-4319-9b67-3acabd58553a None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:07.744325 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d4d9d9-fd3f-4319-9b67-3acabd58553a None admin] PATH_INFO: `/v3/projects/b49e381419e940278bcb6111899589b5/users/b4b26ba3a9e04c38b52db2a46198a53c/roles/ec3751692d9c4ecf81b4a71c936c46be` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:07.746891 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d4d9d9-fd3f-4319-9b67-3acabd58553a None admin] RBAC: Authorizing `identity:create_grant(project_id=b49e381419e940278bcb6111899589b5, user_id=b4b26ba3a9e04c38b52db2a46198a53c, role_id=ec3751692d9c4ecf81b4a71c936c46be)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:07.748349 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d4d9d9-fd3f-4319-9b67-3acabd58553a None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:07.757236 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:07.800609 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf61149-2dd8-4268-8046-f123257173c8 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:07.836599 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5f6ea1-33a9-4b09-ab79-50df8334f8aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:07.837233 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5f6ea1-33a9-4b09-ab79-50df8334f8aa None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:07.837510 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5f6ea1-33a9-4b09-ab79-50df8334f8aa None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:07.837775 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5f6ea1-33a9-4b09-ab79-50df8334f8aa None admin] PATH_INFO: `/v3/services` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:07.838402 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5f6ea1-33a9-4b09-ab79-50df8334f8aa None admin] RBAC: Adding query filter params (type=network) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:07.838699 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5f6ea1-33a9-4b09-ab79-50df8334f8aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:07.840067 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5f6ea1-33a9-4b09-ab79-50df8334f8aa None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:07.852080 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 416/819] 10.222.0.57 () {56 vars in 1112 bytes} [Wed Aug 10 08:24:07 2022] GET /identity/v3/services?type=network => generated 350 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:07.852080 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5f6ea1-33a9-4b09-ab79-50df8334f8aa None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:07.859572 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 404/820] 10.222.0.57 () {58 vars in 1439 bytes} [Wed Aug 10 08:24:07 2022] PUT /identity/v3/projects/b49e381419e940278bcb6111899589b5/users/b4b26ba3a9e04c38b52db2a46198a53c/roles/ec3751692d9c4ecf81b4a71c936c46be => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:24:07.860768 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d4d9d9-fd3f-4319-9b67-3acabd58553a None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:08.082675 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d4d9d9-fd3f-4319-9b67-3acabd58553a None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:08.093413 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5f6ea1-33a9-4b09-ab79-50df8334f8aa None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:08.119907 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae3f2d5-5d7a-4509-87cc-8bc687947712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:08.120963 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae3f2d5-5d7a-4509-87cc-8bc687947712 None admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:08.121451 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae3f2d5-5d7a-4509-87cc-8bc687947712 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:08.121925 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae3f2d5-5d7a-4509-87cc-8bc687947712 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:08.122789 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae3f2d5-5d7a-4509-87cc-8bc687947712 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:08.123977 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:08.124701 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae3f2d5-5d7a-4509-87cc-8bc687947712 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:08.130863 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:08.132894 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa21c03-b4c6-45c4-b13b-e5659ac76ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:08.133990 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa21c03-b4c6-45c4-b13b-e5659ac76ed3 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:08.134373 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa21c03-b4c6-45c4-b13b-e5659ac76ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:08.134746 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa21c03-b4c6-45c4-b13b-e5659ac76ed3 None admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:08.135517 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa21c03-b4c6-45c4-b13b-e5659ac76ed3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:08.135923 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa21c03-b4c6-45c4-b13b-e5659ac76ed3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:08.137428 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa21c03-b4c6-45c4-b13b-e5659ac76ed3 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:08.148018 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 417/821] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Aug 10 08:24:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:08.156845 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa21c03-b4c6-45c4-b13b-e5659ac76ed3 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:08.231037 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 405/822] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 08:24:07 2022] POST /identity/v3/endpoints => generated 337 bytes in 371 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:24:08.272485 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa21c03-b4c6-45c4-b13b-e5659ac76ed3 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:08.317248 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6adf88-44d4-410c-891c-c4a1662a7020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:08.317892 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6adf88-44d4-410c-891c-c4a1662a7020 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:08.318166 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6adf88-44d4-410c-891c-c4a1662a7020 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:08.318432 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6adf88-44d4-410c-891c-c4a1662a7020 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:08.330146 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6adf88-44d4-410c-891c-c4a1662a7020 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:08.332387 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6adf88-44d4-410c-891c-c4a1662a7020 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:08.350018 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-4c6adf88-44d4-410c-891c-c4a1662a7020 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:24:08.350018 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:08.350018 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:24:08.350018 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:24:08.350018 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:24:08.350018 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:24:08.350018 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:24:08.350018 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:24:08.350018 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:24:08.350018 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:24:08.350018 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:24:08.350018 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:24:08.350018 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 08:24:08.350018 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 08:24:08.350018 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 08:24:08.350018 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 08:24:08.351060 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:24:08.351060 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:24:08.351060 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:24:08.351060 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:24:08.351060 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:24:08.351060 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:24:08.351060 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:24:08.351060 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:24:08.351060 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:24:08.351060 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:24:08.351060 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:24:08.351060 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:24:08.351060 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:24:08.351060 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:24:08.351060 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:24:08.351060 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:24:08.351060 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 08:24:08.352568 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 08:24:08.352568 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:24:08.352568 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:24:08.352568 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 418/823] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:24:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 08:24:08.357201 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae3f2d5-5d7a-4509-87cc-8bc687947712 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:08.514668 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae3f2d5-5d7a-4509-87cc-8bc687947712 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:08.553588 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cff13f8-eb56-47d2-85a0-347f3ad965a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:08.554230 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cff13f8-eb56-47d2-85a0-347f3ad965a4 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:08.554474 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cff13f8-eb56-47d2-85a0-347f3ad965a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:08.554742 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cff13f8-eb56-47d2-85a0-347f3ad965a4 None admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:08.555386 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cff13f8-eb56-47d2-85a0-347f3ad965a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:08.555686 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cff13f8-eb56-47d2-85a0-347f3ad965a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:08.557149 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cff13f8-eb56-47d2-85a0-347f3ad965a4 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:08.571065 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 406/824] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Aug 10 08:24:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:08.576123 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6adf88-44d4-410c-891c-c4a1662a7020 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:08.620426 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:08.688444 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6adf88-44d4-410c-891c-c4a1662a7020 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:08.728605 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dfd8d2-b39d-47dd-9292-907cd1a507e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:08.729262 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dfd8d2-b39d-47dd-9292-907cd1a507e4 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:08.729546 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dfd8d2-b39d-47dd-9292-907cd1a507e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:08.729815 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dfd8d2-b39d-47dd-9292-907cd1a507e4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:08.738985 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dfd8d2-b39d-47dd-9292-907cd1a507e4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:08.740437 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dfd8d2-b39d-47dd-9292-907cd1a507e4 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:08.752006 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-29dfd8d2-b39d-47dd-9292-907cd1a507e4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 08:24:08.752006 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:08.752006 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:24:08.752006 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:24:08.752006 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:24:08.752006 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:24:08.752006 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:24:08.752006 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:24:08.752006 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:24:08.752006 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:24:08.752006 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:24:08.752006 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:24:08.752006 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 08:24:08.752006 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 08:24:08.752006 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 08:24:08.752006 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 08:24:08.753660 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:24:08.753660 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:24:08.753660 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 08:24:08.753660 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 08:24:08.753660 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 08:24:08.753660 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 08:24:08.753660 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:24:08.753660 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:24:08.753660 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:24:08.753660 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:24:08.753660 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:24:08.753660 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:24:08.753660 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:24:08.753660 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:24:08.753660 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:24:08.753660 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:24:08.753660 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:24:08.753660 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:24:08.754639 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:24:08.754639 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:24:08.754639 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 08:24:08.754639 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 08:24:08.754639 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 08:24:08.754639 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 08:24:08.754639 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 08:24:08.754639 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 08:24:08.754639 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 08:24:08.754639 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:24:08.754639 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 419/825] 10.222.0.57 () {56 vars in 1093 bytes} [Wed Aug 10 08:24:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 08:24:08.761094 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cff13f8-eb56-47d2-85a0-347f3ad965a4 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:08.828978 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:08.883215 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cff13f8-eb56-47d2-85a0-347f3ad965a4 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:08.920561 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23cd8d2-89d4-455c-98ad-88c434fcd026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:08.921560 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23cd8d2-89d4-455c-98ad-88c434fcd026 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:08.922064 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23cd8d2-89d4-455c-98ad-88c434fcd026 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:08.922585 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23cd8d2-89d4-455c-98ad-88c434fcd026 None admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:08.923534 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23cd8d2-89d4-455c-98ad-88c434fcd026 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:08.924060 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23cd8d2-89d4-455c-98ad-88c434fcd026 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:08.925817 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23cd8d2-89d4-455c-98ad-88c434fcd026 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:08.944224 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 407/826] 10.222.0.57 () {56 vars in 1133 bytes} [Wed Aug 10 08:24:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:08.950079 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dfd8d2-b39d-47dd-9292-907cd1a507e4 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:09.054350 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:09.057907 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dfd8d2-b39d-47dd-9292-907cd1a507e4 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:09.092054 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb1a44b-9179-40a7-9ebf-6c2c38ccb2bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:09.092852 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb1a44b-9179-40a7-9ebf-6c2c38ccb2bb None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:09.093214 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb1a44b-9179-40a7-9ebf-6c2c38ccb2bb None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:09.093624 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb1a44b-9179-40a7-9ebf-6c2c38ccb2bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:09.167495 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb1a44b-9179-40a7-9ebf-6c2c38ccb2bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:09.168944 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb1a44b-9179-40a7-9ebf-6c2c38ccb2bb None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:09.271750 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:09.274865 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2328cc84-fed2-4f8e-b9ed-39148c848a48 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:09.275236 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2328cc84-fed2-4f8e-b9ed-39148c848a48 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:09.275578 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2328cc84-fed2-4f8e-b9ed-39148c848a48 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:09.277259 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 408/827] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:24:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:24:09.278740 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 420/828] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Aug 10 08:24:08 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 08:24:09.285854 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9069e8-c8c4-43fa-934f-40a45ab0017b None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:09.285854 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9069e8-c8c4-43fa-934f-40a45ab0017b None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:09.285854 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9069e8-c8c4-43fa-934f-40a45ab0017b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:09.286097 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb1a44b-9179-40a7-9ebf-6c2c38ccb2bb None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:09.307285 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-2d9069e8-c8c4-43fa-934f-40a45ab0017b None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:24:09.309877 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-2d9069e8-c8c4-43fa-934f-40a45ab0017b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:24:09.397799 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb1a44b-9179-40a7-9ebf-6c2c38ccb2bb None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:09.411078 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 409/829] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:24:09 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 08:24:09.433137 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9069e8-c8c4-43fa-934f-40a45ab0017b None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:09.435322 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5743dae-9936-473a-bacd-c3c53309ca8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:09.435945 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5743dae-9936-473a-bacd-c3c53309ca8c None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:09.436335 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5743dae-9936-473a-bacd-c3c53309ca8c None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:09.436608 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5743dae-9936-473a-bacd-c3c53309ca8c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:09.447858 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5743dae-9936-473a-bacd-c3c53309ca8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:09.449469 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5743dae-9936-473a-bacd-c3c53309ca8c None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:09.465524 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-c5743dae-9936-473a-bacd-c3c53309ca8c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:24:09.465524 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:09.465524 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:24:09.465524 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:24:09.465524 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:24:09.465524 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:24:09.465524 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:24:09.465524 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:24:09.465524 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:24:09.465524 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:24:09.465524 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:24:09.465524 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:24:09.465524 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 08:24:09.465524 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 08:24:09.465524 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 08:24:09.465524 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 08:24:09.466505 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:24:09.466505 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:24:09.466505 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:24:09.466505 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:24:09.466505 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:24:09.466505 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:24:09.466505 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:24:09.466505 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:24:09.466505 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:24:09.466505 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:24:09.466505 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:24:09.466505 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:24:09.466505 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:24:09.466505 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:24:09.466505 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:24:09.466505 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:24:09.466505 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 08:24:09.467439 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 08:24:09.467439 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:24:09.467439 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:24:09.467439 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 421/830] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:24:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:24:09.473319 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5743dae-9936-473a-bacd-c3c53309ca8c None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:09.477920 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:09.525385 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:09.548436 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9069e8-c8c4-43fa-934f-40a45ab0017b None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:09.579788 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5743dae-9936-473a-bacd-c3c53309ca8c None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:09.586230 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0f6338-042e-4fbe-8dff-def9b769ccd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:09.587494 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0f6338-042e-4fbe-8dff-def9b769ccd7 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:09.587841 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0f6338-042e-4fbe-8dff-def9b769ccd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:09.588119 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0f6338-042e-4fbe-8dff-def9b769ccd7 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:09.588703 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0f6338-042e-4fbe-8dff-def9b769ccd7 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:09.590103 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0f6338-042e-4fbe-8dff-def9b769ccd7 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:09.599689 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-0f0f6338-042e-4fbe-8dff-def9b769ccd7 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 08:24:09.599689 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:09.599689 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 08:24:09.599689 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 08:24:09.599689 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 08:24:09.599689 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 08:24:09.599689 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 08:24:09.599689 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 08:24:09.599689 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 08:24:09.599689 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:24:09.599689 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 08:24:09.599689 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:24:09.599689 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:09.599689 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:24:09.599689 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:24:09.599689 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:24:09.599689 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:24:09.600699 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:24:09.600699 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:24:09.600699 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:24:09.600699 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:24:09.600699 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:24:09.600699 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:24:09.600699 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 08:24:09.600699 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 08:24:09.600699 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 08:24:09.600699 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 08:24:09.600699 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:24:09.600699 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:24:09.600699 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:24:09.600699 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:24:09.600699 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:24:09.600699 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:24:09.607039 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:24:09.607039 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:24:09.607039 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:24:09.607039 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:24:09.607039 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:24:09.607039 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:24:09.607039 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:24:09.607039 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:24:09.607039 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:24:09.607039 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:24:09.607039 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 08:24:09.607039 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 08:24:09.607039 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 08:24:09.607039 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:24:09.607039 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 410/831] 10.222.0.57 () {56 vars in 1114 bytes} [Wed Aug 10 08:24:09 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:24:09.607840 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c0dc3a-5e4b-4d22-9d4c-e638b5ef95a2 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:09.607840 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c0dc3a-5e4b-4d22-9d4c-e638b5ef95a2 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:09.607840 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c0dc3a-5e4b-4d22-9d4c-e638b5ef95a2 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:09.607840 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 411/832] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:24:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:24:09.609002 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c0dc3a-5e4b-4d22-9d4c-e638b5ef95a2 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:09.621849 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d1f728-7f96-4f83-b514-a64cb4b53dfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:09.622461 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d1f728-7f96-4f83-b514-a64cb4b53dfb None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:09.622743 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d1f728-7f96-4f83-b514-a64cb4b53dfb None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:09.623016 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d1f728-7f96-4f83-b514-a64cb4b53dfb None admin] PATH_INFO: `/v3/domains` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:09.623666 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d1f728-7f96-4f83-b514-a64cb4b53dfb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:09.623969 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d1f728-7f96-4f83-b514-a64cb4b53dfb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:09.625370 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d1f728-7f96-4f83-b514-a64cb4b53dfb None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:09.637321 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 422/833] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Aug 10 08:24:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:24:09.638499 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ddeaf2-f664-4ea5-a718-e48cd4eaff77 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:09.638783 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ddeaf2-f664-4ea5-a718-e48cd4eaff77 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:09.639053 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ddeaf2-f664-4ea5-a718-e48cd4eaff77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:09.657154 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-c0ddeaf2-f664-4ea5-a718-e48cd4eaff77 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:24:09.658338 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-c0ddeaf2-f664-4ea5-a718-e48cd4eaff77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:24:09.722626 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c0dc3a-5e4b-4d22-9d4c-e638b5ef95a2 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:09.724074 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:09.737222 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 423/834] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:24:09 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 08:24:09.737222 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ddeaf2-f664-4ea5-a718-e48cd4eaff77 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:09.771235 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaeed09-9e4f-4d6d-9490-86a35328eb8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:09.771849 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaeed09-9e4f-4d6d-9490-86a35328eb8d None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:09.772125 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaeed09-9e4f-4d6d-9490-86a35328eb8d None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:09.772398 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaeed09-9e4f-4d6d-9490-86a35328eb8d None admin] PATH_INFO: `/v3/services` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:09.773221 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaeed09-9e4f-4d6d-9490-86a35328eb8d None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:09.773526 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaeed09-9e4f-4d6d-9490-86a35328eb8d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:09.775668 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaeed09-9e4f-4d6d-9490-86a35328eb8d None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:09.780851 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:09.785075 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 412/835] 10.222.0.57 () {56 vars in 1114 bytes} [Wed Aug 10 08:24:09 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:09.786038 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaeed09-9e4f-4d6d-9490-86a35328eb8d None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:09.844590 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ddeaf2-f664-4ea5-a718-e48cd4eaff77 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:09.882092 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a982f070-b46d-4599-81a9-cdeaca5ea382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:09.882723 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a982f070-b46d-4599-81a9-cdeaca5ea382 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:09.883095 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a982f070-b46d-4599-81a9-cdeaca5ea382 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:09.883289 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a982f070-b46d-4599-81a9-cdeaca5ea382 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:09.890356 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a982f070-b46d-4599-81a9-cdeaca5ea382 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:09.891790 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a982f070-b46d-4599-81a9-cdeaca5ea382 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:09.894574 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaeed09-9e4f-4d6d-9490-86a35328eb8d None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:09.903244 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-a982f070-b46d-4599-81a9-cdeaca5ea382 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 08:24:09.903244 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:09.903244 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:24:09.903244 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:24:09.903244 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:24:09.903244 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:24:09.903244 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:24:09.903244 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:24:09.903244 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:24:09.903244 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:24:09.903244 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:24:09.903244 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:24:09.903244 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 08:24:09.903244 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 08:24:09.903244 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 08:24:09.904191 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 08:24:09.904191 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:24:09.904191 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:24:09.904191 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:24:09.904191 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:24:09.904191 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:24:09.904191 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:24:09.904191 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:24:09.904191 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:24:09.904191 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:24:09.904191 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:24:09.904191 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:24:09.904191 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:24:09.904191 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:24:09.904191 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:24:09.904191 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:24:09.904191 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:24:09.905148 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 08:24:09.905148 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 08:24:09.905148 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 08:24:09.905148 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 08:24:09.905148 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 08:24:09.905148 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 08:24:09.905148 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 08:24:09.905148 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:24:09.909501 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 424/836] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Aug 10 08:24:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:24:09.910264 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a982f070-b46d-4599-81a9-cdeaca5ea382 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:09.934387 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618eb3d0-92fe-41f2-82d0-9463186d2e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:09.935042 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618eb3d0-92fe-41f2-82d0-9463186d2e4d None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:09.935302 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618eb3d0-92fe-41f2-82d0-9463186d2e4d None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:09.935572 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618eb3d0-92fe-41f2-82d0-9463186d2e4d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:09.946880 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618eb3d0-92fe-41f2-82d0-9463186d2e4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 08:24:09.951576 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-618eb3d0-92fe-41f2-82d0-9463186d2e4d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 08:24:09.951576 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:09.951576 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:24:09.951576 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:24:09.951576 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:24:09.951576 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:24:09.951576 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:24:09.951576 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:24:09.951576 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:24:09.951576 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:24:09.951576 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:24:09.951576 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:24:09.951576 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 08:24:09.951576 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 08:24:09.951576 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 08:24:09.951576 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 08:24:09.952520 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 08:24:09.952520 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 08:24:09.952520 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 08:24:09.952520 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 08:24:09.952520 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:24:09.952520 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:24:09.952520 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:24:09.952520 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:24:09.952520 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:24:09.952520 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:24:09.952520 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:24:09.952520 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:24:09.952520 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:24:09.952520 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:24:09.952520 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:24:09.952520 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:24:09.953446 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:24:09.953446 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:24:09.953446 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:24:09.953446 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:24:09.953446 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 08:24:09.953446 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 08:24:09.953446 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 08:24:09.953446 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 08:24:09.953446 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 08:24:09.953446 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 08:24:09.953446 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 08:24:09.953446 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:24:09.953446 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 413/837] 10.222.0.57 () {56 vars in 1102 bytes} [Wed Aug 10 08:24:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:24:09.954469 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618eb3d0-92fe-41f2-82d0-9463186d2e4d None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:09.984438 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:10.015189 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:10.018306 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a982f070-b46d-4599-81a9-cdeaca5ea382 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:10.055537 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820b574c-5770-4442-996a-ea7ad7c08ce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:10.056607 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820b574c-5770-4442-996a-ea7ad7c08ce9 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:10.057175 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820b574c-5770-4442-996a-ea7ad7c08ce9 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:10.057450 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820b574c-5770-4442-996a-ea7ad7c08ce9 None admin] PATH_INFO: `/v3/services` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:10.058322 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820b574c-5770-4442-996a-ea7ad7c08ce9 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:10.058631 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820b574c-5770-4442-996a-ea7ad7c08ce9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:10.060642 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820b574c-5770-4442-996a-ea7ad7c08ce9 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:10.061667 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618eb3d0-92fe-41f2-82d0-9463186d2e4d None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:10.066692 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 425/838] 10.222.0.57 () {56 vars in 1114 bytes} [Wed Aug 10 08:24:09 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:10.067725 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820b574c-5770-4442-996a-ea7ad7c08ce9 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:10.113323 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71722cc-ec88-42a7-9fe6-53d24d098552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:10.113979 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71722cc-ec88-42a7-9fe6-53d24d098552 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:10.114256 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71722cc-ec88-42a7-9fe6-53d24d098552 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:10.114524 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71722cc-ec88-42a7-9fe6-53d24d098552 None admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:10.115233 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71722cc-ec88-42a7-9fe6-53d24d098552 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:10.116258 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71722cc-ec88-42a7-9fe6-53d24d098552 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:10.117766 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71722cc-ec88-42a7-9fe6-53d24d098552 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:10.136003 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 414/839] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Aug 10 08:24:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:10.141730 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71722cc-ec88-42a7-9fe6-53d24d098552 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:10.188183 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820b574c-5770-4442-996a-ea7ad7c08ce9 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:10.229337 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dec1f0-ef41-4e5f-b492-8b79ff27aa2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:10.230259 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dec1f0-ef41-4e5f-b492-8b79ff27aa2a None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:10.230858 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dec1f0-ef41-4e5f-b492-8b79ff27aa2a None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:10.231311 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dec1f0-ef41-4e5f-b492-8b79ff27aa2a None admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:10.232667 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dec1f0-ef41-4e5f-b492-8b79ff27aa2a None admin] RBAC: Adding query filter params (name=service) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:10.233464 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dec1f0-ef41-4e5f-b492-8b79ff27aa2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:10.235308 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dec1f0-ef41-4e5f-b492-8b79ff27aa2a None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:10.247155 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 426/840] 10.222.0.57 () {56 vars in 1103 bytes} [Wed Aug 10 08:24:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:10.248413 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1db6a67-4cb3-4319-8468-0a1fc25c08e1 None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:10.248702 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1db6a67-4cb3-4319-8468-0a1fc25c08e1 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:10.249010 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1db6a67-4cb3-4319-8468-0a1fc25c08e1 None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:10.250346 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 427/841] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:24:10 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:24:10.253974 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1db6a67-4cb3-4319-8468-0a1fc25c08e1 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:10.260880 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:10.276478 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71722cc-ec88-42a7-9fe6-53d24d098552 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:10.293267 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:10.314590 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a009ff-9ae6-4856-8e6f-c7f49d9fdb64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:10.315248 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a009ff-9ae6-4856-8e6f-c7f49d9fdb64 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:10.315530 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a009ff-9ae6-4856-8e6f-c7f49d9fdb64 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:10.315805 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a009ff-9ae6-4856-8e6f-c7f49d9fdb64 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:10.316534 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a009ff-9ae6-4856-8e6f-c7f49d9fdb64 None admin] RBAC: Adding query filter params (role.id=ec3751692d9c4ecf81b4a71c936c46be, scope.project.id=b49e381419e940278bcb6111899589b5, user.id=a943008193914f76a663d489bd73e873) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:10.316842 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a009ff-9ae6-4856-8e6f-c7f49d9fdb64 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:10.318309 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a009ff-9ae6-4856-8e6f-c7f49d9fdb64 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:10.333208 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 415/842] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Aug 10 08:24:10 2022] GET /identity/v3/role_assignments?user.id=a943008193914f76a663d489bd73e873&scope.project.id=b49e381419e940278bcb6111899589b5&role.id=ec3751692d9c4ecf81b4a71c936c46be => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:24:10.333208 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d760bca-d93c-4e4b-b184-74e21cd7f50b None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:10.333208 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d760bca-d93c-4e4b-b184-74e21cd7f50b None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:10.333208 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d760bca-d93c-4e4b-b184-74e21cd7f50b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:10.348023 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-7d760bca-d93c-4e4b-b184-74e21cd7f50b None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:24:10.349300 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-7d760bca-d93c-4e4b-b184-74e21cd7f50b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:24:10.382049 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1db6a67-4cb3-4319-8468-0a1fc25c08e1 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:10.441348 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18131fb5-7b02-48a2-8b85-7fd5506531cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:10.442145 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18131fb5-7b02-48a2-8b85-7fd5506531cd None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:10.442524 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18131fb5-7b02-48a2-8b85-7fd5506531cd None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:10.442912 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18131fb5-7b02-48a2-8b85-7fd5506531cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:10.447355 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 416/843] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:24:10 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 08:24:10.456040 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18131fb5-7b02-48a2-8b85-7fd5506531cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:10.457715 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18131fb5-7b02-48a2-8b85-7fd5506531cd None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:10.472105 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d760bca-d93c-4e4b-b184-74e21cd7f50b None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:10.478715 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-18131fb5-7b02-48a2-8b85-7fd5506531cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:24:10.478715 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:10.478715 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:24:10.478715 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:24:10.478715 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:24:10.478715 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:24:10.478715 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:24:10.478715 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:24:10.478715 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:24:10.478715 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:24:10.478715 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:24:10.478715 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:24:10.478715 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 08:24:10.478715 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 08:24:10.478715 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 08:24:10.478715 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 08:24:10.479730 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:24:10.479730 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:24:10.479730 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:24:10.479730 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:24:10.479730 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:24:10.479730 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:24:10.479730 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:24:10.479730 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:24:10.479730 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:24:10.479730 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:24:10.479730 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:24:10.479730 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:24:10.479730 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:24:10.479730 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:24:10.479730 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:24:10.479730 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:24:10.479730 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 08:24:10.480660 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 08:24:10.480660 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:24:10.480660 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:24:10.480660 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 428/844] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:24:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:24:10.483102 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18131fb5-7b02-48a2-8b85-7fd5506531cd None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:10.573141 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:10.574615 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:10.603756 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18131fb5-7b02-48a2-8b85-7fd5506531cd None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:10.608858 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d760bca-d93c-4e4b-b184-74e21cd7f50b None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:10.649899 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138c482a-c920-48fb-af43-b9b446ba4ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:10.650514 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138c482a-c920-48fb-af43-b9b446ba4ccf None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:10.650794 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138c482a-c920-48fb-af43-b9b446ba4ccf None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:10.651069 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138c482a-c920-48fb-af43-b9b446ba4ccf None admin] PATH_INFO: `/v3/domains` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:10.651711 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138c482a-c920-48fb-af43-b9b446ba4ccf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:10.652017 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138c482a-c920-48fb-af43-b9b446ba4ccf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:10.653424 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138c482a-c920-48fb-af43-b9b446ba4ccf None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:10.654436 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3ce424-0002-42f0-9886-5f53fe3e79a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:10.655588 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3ce424-0002-42f0-9886-5f53fe3e79a3 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:10.656633 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3ce424-0002-42f0-9886-5f53fe3e79a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:10.657145 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3ce424-0002-42f0-9886-5f53fe3e79a3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:10.666858 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 429/845] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Aug 10 08:24:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:10.670952 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3ce424-0002-42f0-9886-5f53fe3e79a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 08:24:10.671951 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138c482a-c920-48fb-af43-b9b446ba4ccf None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:10.677909 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-1a3ce424-0002-42f0-9886-5f53fe3e79a3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 08:24:10.677909 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:10.677909 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:24:10.677909 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:24:10.677909 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:24:10.677909 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:24:10.677909 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:24:10.677909 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:24:10.677909 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:24:10.677909 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:24:10.677909 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:24:10.677909 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:24:10.677909 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 08:24:10.677909 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 08:24:10.677909 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 08:24:10.677909 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 08:24:10.678902 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 08:24:10.678902 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 08:24:10.678902 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 08:24:10.678902 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 08:24:10.678902 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:24:10.678902 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:24:10.678902 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:24:10.678902 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:24:10.678902 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:24:10.678902 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:24:10.678902 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:24:10.678902 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:24:10.678902 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:24:10.678902 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:24:10.678902 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:24:10.678902 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:24:10.679778 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:24:10.679778 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:24:10.679778 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:24:10.679778 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:24:10.679778 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 08:24:10.679778 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 08:24:10.679778 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 08:24:10.679778 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 08:24:10.679778 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 08:24:10.679778 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 08:24:10.679778 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 08:24:10.679778 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:24:10.682317 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 417/846] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Aug 10 08:24:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 08:24:10.688193 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3ce424-0002-42f0-9886-5f53fe3e79a3 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:10.807843 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138c482a-c920-48fb-af43-b9b446ba4ccf None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:10.811757 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3ce424-0002-42f0-9886-5f53fe3e79a3 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:10.848187 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2856dc-2f2b-4299-a18d-1fe91cee87e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:10.848804 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2856dc-2f2b-4299-a18d-1fe91cee87e0 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:10.849219 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2856dc-2f2b-4299-a18d-1fe91cee87e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:10.849557 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2856dc-2f2b-4299-a18d-1fe91cee87e0 None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:10.850359 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2856dc-2f2b-4299-a18d-1fe91cee87e0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:10.850641 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2856dc-2f2b-4299-a18d-1fe91cee87e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:10.852198 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2856dc-2f2b-4299-a18d-1fe91cee87e0 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:10.856437 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1502b210-e776-4d20-ac32-a278c37f6020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:10.857317 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1502b210-e776-4d20-ac32-a278c37f6020 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:10.858351 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1502b210-e776-4d20-ac32-a278c37f6020 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:10.858812 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1502b210-e776-4d20-ac32-a278c37f6020 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:10.867272 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 418/847] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Aug 10 08:24:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:24:10.868324 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2856dc-2f2b-4299-a18d-1fe91cee87e0 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:10.873822 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1502b210-e776-4d20-ac32-a278c37f6020 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:10.875262 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1502b210-e776-4d20-ac32-a278c37f6020 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:10.875524 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:10.877534 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:10.887928 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-1502b210-e776-4d20-ac32-a278c37f6020 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 08:24:10.887928 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:10.887928 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:24:10.887928 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:24:10.887928 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:24:10.887928 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:24:10.887928 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:24:10.887928 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:24:10.887928 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:24:10.887928 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:24:10.887928 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:24:10.887928 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:24:10.887928 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 08:24:10.887928 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 08:24:10.887928 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 08:24:10.887928 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 08:24:10.891730 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:24:10.891730 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:24:10.891730 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 08:24:10.891730 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 08:24:10.891730 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 08:24:10.891730 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 08:24:10.891730 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:24:10.891730 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:24:10.891730 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:24:10.891730 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:24:10.891730 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:24:10.891730 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:24:10.891730 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:24:10.891730 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:24:10.891730 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:24:10.891730 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:24:10.891730 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:24:10.891730 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:24:10.892688 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:24:10.892688 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:24:10.892688 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 08:24:10.892688 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 08:24:10.892688 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 08:24:10.892688 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 08:24:10.892688 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 08:24:10.892688 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 08:24:10.892688 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 08:24:10.892688 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:24:10.893551 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 430/848] 10.222.0.57 () {56 vars in 1117 bytes} [Wed Aug 10 08:24:10 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:24:10.896777 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1502b210-e776-4d20-ac32-a278c37f6020 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:11.004002 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1502b210-e776-4d20-ac32-a278c37f6020 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:11.013739 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2856dc-2f2b-4299-a18d-1fe91cee87e0 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:11.039331 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb8558b-4c2c-4e91-99de-0865f8e2b05a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:11.040021 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb8558b-4c2c-4e91-99de-0865f8e2b05a None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:11.040366 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb8558b-4c2c-4e91-99de-0865f8e2b05a None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:11.040706 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb8558b-4c2c-4e91-99de-0865f8e2b05a None admin] PATH_INFO: `/v3/users` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:11.041555 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb8558b-4c2c-4e91-99de-0865f8e2b05a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:11.041945 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb8558b-4c2c-4e91-99de-0865f8e2b05a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:11.043484 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb8558b-4c2c-4e91-99de-0865f8e2b05a None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:11.061025 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4233d471-5e8f-47ca-8a31-dab4ef632110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:11.061899 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4233d471-5e8f-47ca-8a31-dab4ef632110 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:11.062354 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4233d471-5e8f-47ca-8a31-dab4ef632110 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:11.062835 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4233d471-5e8f-47ca-8a31-dab4ef632110 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:11.066607 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 431/849] 10.222.0.57 () {56 vars in 1149 bytes} [Wed Aug 10 08:24:10 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:11.068305 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb8558b-4c2c-4e91-99de-0865f8e2b05a None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:11.080407 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4233d471-5e8f-47ca-8a31-dab4ef632110 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:11.082561 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4233d471-5e8f-47ca-8a31-dab4ef632110 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:11.098660 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-4233d471-5e8f-47ca-8a31-dab4ef632110 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:24:11.098660 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:11.098660 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:24:11.098660 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:24:11.098660 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:24:11.098660 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:24:11.098660 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:24:11.098660 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:24:11.098660 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:24:11.098660 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:24:11.098660 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:24:11.098660 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:24:11.098660 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 08:24:11.098660 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 08:24:11.098660 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 08:24:11.098660 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 08:24:11.099629 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:24:11.099629 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:24:11.099629 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:24:11.099629 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:24:11.099629 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:24:11.099629 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:24:11.099629 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:24:11.099629 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:24:11.099629 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:24:11.099629 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:24:11.099629 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:24:11.099629 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:24:11.099629 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:24:11.099629 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:24:11.099629 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:24:11.099629 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:24:11.099629 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 08:24:11.100550 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 08:24:11.100550 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:24:11.100550 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:24:11.104545 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 419/850] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:24:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:24:11.105840 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4233d471-5e8f-47ca-8a31-dab4ef632110 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:11.126927 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:11.172815 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb8558b-4c2c-4e91-99de-0865f8e2b05a None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:11.181298 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:11.214790 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f10d59b-5906-4825-bc9e-7cdea52711b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:11.215412 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f10d59b-5906-4825-bc9e-7cdea52711b9 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:11.215744 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f10d59b-5906-4825-bc9e-7cdea52711b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:11.216030 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f10d59b-5906-4825-bc9e-7cdea52711b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:11.217554 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4233d471-5e8f-47ca-8a31-dab4ef632110 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:11.256415 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7e3ed0-e7be-490a-8dbd-1d2af6be14f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:11.257464 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7e3ed0-e7be-490a-8dbd-1d2af6be14f5 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:11.257921 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7e3ed0-e7be-490a-8dbd-1d2af6be14f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:11.258398 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7e3ed0-e7be-490a-8dbd-1d2af6be14f5 None admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:11.259247 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7e3ed0-e7be-490a-8dbd-1d2af6be14f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:11.259736 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7e3ed0-e7be-490a-8dbd-1d2af6be14f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:11.261503 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7e3ed0-e7be-490a-8dbd-1d2af6be14f5 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:11.280790 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 420/851] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Aug 10 08:24:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:24:11.282551 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7e3ed0-e7be-490a-8dbd-1d2af6be14f5 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:11.292579 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f10d59b-5906-4825-bc9e-7cdea52711b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:11.294142 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f10d59b-5906-4825-bc9e-7cdea52711b9 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:11.351976 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:11.396230 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7e3ed0-e7be-490a-8dbd-1d2af6be14f5 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:11.398308 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 432/852] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Aug 10 08:24:11 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 10 08:24:11.405735 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f10d59b-5906-4825-bc9e-7cdea52711b9 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:11.413973 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:11.440535 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0e5f86-de0e-43c8-8e4a-ebb8e2df1b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:11.441697 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0e5f86-de0e-43c8-8e4a-ebb8e2df1b7c None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:11.442165 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0e5f86-de0e-43c8-8e4a-ebb8e2df1b7c None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:11.442542 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0e5f86-de0e-43c8-8e4a-ebb8e2df1b7c None admin] PATH_INFO: `/v3/users/swift` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:11.450116 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0e5f86-de0e-43c8-8e4a-ebb8e2df1b7c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:11.452729 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0e5f86-de0e-43c8-8e4a-ebb8e2df1b7c None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:11.465878 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-3b0e5f86-de0e-43c8-8e4a-ebb8e2df1b7c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 10 08:24:11.465878 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:11.465878 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:24:11.465878 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:24:11.465878 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:24:11.465878 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:24:11.465878 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:24:11.465878 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:24:11.465878 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:24:11.465878 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:24:11.465878 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:24:11.465878 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:24:11.465878 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 08:24:11.465878 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 08:24:11.465878 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 08:24:11.465878 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 08:24:11.467066 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:24:11.467066 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:24:11.467066 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 08:24:11.467066 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 08:24:11.467066 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 08:24:11.467066 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 08:24:11.467066 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:24:11.467066 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:24:11.467066 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:24:11.467066 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:24:11.467066 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:24:11.467066 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:24:11.467066 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:24:11.467066 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:24:11.467066 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:24:11.467066 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:24:11.467066 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:24:11.467066 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:24:11.468023 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:24:11.468023 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:24:11.468023 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 08:24:11.468023 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 08:24:11.468023 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 08:24:11.468023 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 08:24:11.468023 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 08:24:11.468023 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 08:24:11.468023 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 10 08:24:11.468023 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:24:11.473853 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 421/853] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Aug 10 08:24:11 2022] GET /identity/v3/users/swift => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 08:24:11.475146 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0e5f86-de0e-43c8-8e4a-ebb8e2df1b7c None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:11.514744 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f10d59b-5906-4825-bc9e-7cdea52711b9 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:11.552400 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39ad8e1-bb33-485a-99f0-1617159a7e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:11.553388 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39ad8e1-bb33-485a-99f0-1617159a7e79 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:11.553854 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39ad8e1-bb33-485a-99f0-1617159a7e79 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:11.554307 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39ad8e1-bb33-485a-99f0-1617159a7e79 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:11.565080 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39ad8e1-bb33-485a-99f0-1617159a7e79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:11.566851 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39ad8e1-bb33-485a-99f0-1617159a7e79 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:11.583523 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-b39ad8e1-bb33-485a-99f0-1617159a7e79 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:24:11.583523 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:11.583523 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:24:11.583523 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:24:11.583523 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:24:11.583523 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:24:11.583523 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:24:11.583523 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:24:11.583523 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:24:11.583523 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:24:11.583523 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:24:11.583523 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:24:11.583523 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 08:24:11.583523 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 08:24:11.583523 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 08:24:11.583523 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 08:24:11.584502 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:24:11.584502 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:24:11.584502 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:24:11.584502 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:24:11.584502 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:24:11.584502 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:24:11.584502 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:24:11.584502 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:24:11.584502 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:24:11.584502 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:24:11.584502 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:24:11.584502 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:24:11.584502 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:24:11.584502 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:24:11.584502 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:24:11.584502 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:24:11.584502 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 08:24:11.585454 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 08:24:11.585454 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:24:11.585454 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:24:11.586593 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0e5f86-de0e-43c8-8e4a-ebb8e2df1b7c None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:11.587821 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 433/854] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:24:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:24:11.592780 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39ad8e1-bb33-485a-99f0-1617159a7e79 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:11.605936 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:11.627138 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a48d8d-a2d5-463a-9542-ea603ca2fd4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:11.629619 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a48d8d-a2d5-463a-9542-ea603ca2fd4a None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:11.629914 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a48d8d-a2d5-463a-9542-ea603ca2fd4a None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:11.630188 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a48d8d-a2d5-463a-9542-ea603ca2fd4a None admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:11.631260 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a48d8d-a2d5-463a-9542-ea603ca2fd4a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:11.631575 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a48d8d-a2d5-463a-9542-ea603ca2fd4a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:11.633099 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a48d8d-a2d5-463a-9542-ea603ca2fd4a None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:11.665006 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 422/855] 10.222.0.57 () {56 vars in 1135 bytes} [Wed Aug 10 08:24:11 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:24:11.667387 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a48d8d-a2d5-463a-9542-ea603ca2fd4a None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:11.707236 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:11.716458 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39ad8e1-bb33-485a-99f0-1617159a7e79 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:11.751613 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5b0f52-3016-4e47-a35e-fe96a9a72760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:11.753035 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5b0f52-3016-4e47-a35e-fe96a9a72760 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:11.753468 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5b0f52-3016-4e47-a35e-fe96a9a72760 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:11.753942 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5b0f52-3016-4e47-a35e-fe96a9a72760 None admin] PATH_INFO: `/v3/domains` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:11.754735 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5b0f52-3016-4e47-a35e-fe96a9a72760 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:11.755187 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5b0f52-3016-4e47-a35e-fe96a9a72760 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:11.756697 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5b0f52-3016-4e47-a35e-fe96a9a72760 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:11.769766 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 434/856] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Aug 10 08:24:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:24:11.777878 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5b0f52-3016-4e47-a35e-fe96a9a72760 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:11.787536 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a48d8d-a2d5-463a-9542-ea603ca2fd4a None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:11.827641 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d69880a-76ee-4218-816c-7b117d8d0f08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:11.829142 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d69880a-76ee-4218-816c-7b117d8d0f08 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:11.829647 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d69880a-76ee-4218-816c-7b117d8d0f08 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:11.830136 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d69880a-76ee-4218-816c-7b117d8d0f08 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:11.859514 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:11.895480 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5b0f52-3016-4e47-a35e-fe96a9a72760 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:11.919280 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d69880a-76ee-4218-816c-7b117d8d0f08 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:11.921273 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d69880a-76ee-4218-816c-7b117d8d0f08 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:11.935577 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f719d79c-fe8a-4838-8035-94472a8dff08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:11.936463 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f719d79c-fe8a-4838-8035-94472a8dff08 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:11.936978 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f719d79c-fe8a-4838-8035-94472a8dff08 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:11.937437 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f719d79c-fe8a-4838-8035-94472a8dff08 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:11.945111 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f719d79c-fe8a-4838-8035-94472a8dff08 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:11.946784 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f719d79c-fe8a-4838-8035-94472a8dff08 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:11.953792 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:11.959571 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-f719d79c-fe8a-4838-8035-94472a8dff08 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 08:24:11.959571 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:11.959571 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:24:11.959571 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:24:11.959571 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:24:11.959571 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:24:11.959571 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:24:11.959571 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:24:11.959571 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:24:11.959571 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:24:11.959571 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:24:11.959571 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:24:11.959571 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 08:24:11.959571 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 08:24:11.959571 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 08:24:11.969034 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 08:24:11.969034 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:24:11.969034 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:24:11.969034 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:24:11.969034 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:24:11.969034 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:24:11.969034 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:24:11.969034 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:24:11.969034 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:24:11.969034 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:24:11.969034 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:24:11.969034 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:24:11.969034 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:24:11.969034 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:24:11.969034 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:24:11.969034 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:24:11.969034 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:24:11.969963 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 08:24:11.969963 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 08:24:11.969963 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 08:24:11.969963 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 08:24:11.969963 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 08:24:11.969963 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 08:24:11.969963 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 08:24:11.969963 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:24:11.969963 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 435/857] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Aug 10 08:24:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:24:11.972327 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f719d79c-fe8a-4838-8035-94472a8dff08 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:12.031426 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 423/858] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Aug 10 08:24:11 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 08:24:12.039680 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d69880a-76ee-4218-816c-7b117d8d0f08 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:12.101747 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f719d79c-fe8a-4838-8035-94472a8dff08 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:12.133355 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:12.141246 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a08d357-53bd-4c7e-a5b4-3a51ae816a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:12.141851 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a08d357-53bd-4c7e-a5b4-3a51ae816a73 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:12.142127 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a08d357-53bd-4c7e-a5b4-3a51ae816a73 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:12.149121 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a08d357-53bd-4c7e-a5b4-3a51ae816a73 None admin] PATH_INFO: `/v3/projects` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:12.149121 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a08d357-53bd-4c7e-a5b4-3a51ae816a73 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:12.149121 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a08d357-53bd-4c7e-a5b4-3a51ae816a73 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:12.149121 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a08d357-53bd-4c7e-a5b4-3a51ae816a73 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:12.158182 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 436/859] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Aug 10 08:24:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:24:12.164619 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a08d357-53bd-4c7e-a5b4-3a51ae816a73 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:12.177926 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d69880a-76ee-4218-816c-7b117d8d0f08 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:12.218936 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:12.224386 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12a4cbd-43dd-4ee2-90b1-b52035ffa1c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:12.225154 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12a4cbd-43dd-4ee2-90b1-b52035ffa1c9 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:12.225486 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12a4cbd-43dd-4ee2-90b1-b52035ffa1c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:12.225843 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12a4cbd-43dd-4ee2-90b1-b52035ffa1c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:12.235826 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12a4cbd-43dd-4ee2-90b1-b52035ffa1c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:12.238111 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12a4cbd-43dd-4ee2-90b1-b52035ffa1c9 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:12.254332 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-e12a4cbd-43dd-4ee2-90b1-b52035ffa1c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:24:12.254332 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:12.254332 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:24:12.254332 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:24:12.254332 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:24:12.254332 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:24:12.254332 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:24:12.254332 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:24:12.254332 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:24:12.254332 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:24:12.254332 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:24:12.254332 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:24:12.254332 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 08:24:12.254332 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 08:24:12.254332 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 08:24:12.254332 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 08:24:12.255523 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:24:12.255523 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:24:12.255523 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:24:12.255523 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:24:12.255523 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:24:12.255523 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:24:12.255523 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:24:12.255523 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:24:12.255523 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:24:12.255523 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:24:12.255523 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:24:12.255523 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:24:12.255523 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:24:12.255523 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:24:12.255523 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:24:12.255523 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:24:12.255523 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 08:24:12.256883 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 08:24:12.256883 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:24:12.256883 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:24:12.256883 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 424/860] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:24:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:24:12.256883 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5e2acc-ae11-4147-a1af-72f467c90b22 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:12.257225 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5e2acc-ae11-4147-a1af-72f467c90b22 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:12.257225 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5e2acc-ae11-4147-a1af-72f467c90b22 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:12.262847 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 425/861] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:24:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:24:12.262847 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5e2acc-ae11-4147-a1af-72f467c90b22 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:12.279880 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a08d357-53bd-4c7e-a5b4-3a51ae816a73 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:12.316307 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcaae0b-3355-4900-aa78-8b8c91018352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:12.317851 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcaae0b-3355-4900-aa78-8b8c91018352 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:12.318566 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcaae0b-3355-4900-aa78-8b8c91018352 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:12.318988 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcaae0b-3355-4900-aa78-8b8c91018352 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:12.319860 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcaae0b-3355-4900-aa78-8b8c91018352 None admin] RBAC: Adding query filter params (role.id=73fdd60e2f804ee69b76fa9145a472ff, scope.project.id=b49e381419e940278bcb6111899589b5, user.id=c140d31bd2b84c48a170e3ca90a041ce) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:12.320306 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcaae0b-3355-4900-aa78-8b8c91018352 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:12.322620 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcaae0b-3355-4900-aa78-8b8c91018352 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:12.330888 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 437/862] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Aug 10 08:24:12 2022] GET /identity/v3/role_assignments?user.id=c140d31bd2b84c48a170e3ca90a041ce&scope.project.id=b49e381419e940278bcb6111899589b5&role.id=73fdd60e2f804ee69b76fa9145a472ff => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:24:12.332365 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacd6402-9c7e-465b-9b2f-60e10e5dc44f None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:12.332892 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacd6402-9c7e-465b-9b2f-60e10e5dc44f None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:12.333454 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacd6402-9c7e-465b-9b2f-60e10e5dc44f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:12.359639 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-cacd6402-9c7e-465b-9b2f-60e10e5dc44f None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:24:12.360838 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-cacd6402-9c7e-465b-9b2f-60e10e5dc44f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:24:12.385327 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5e2acc-ae11-4147-a1af-72f467c90b22 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:12.430262 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:12.436876 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42fc28d-1120-40f7-98a8-27828fba3973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:12.438314 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42fc28d-1120-40f7-98a8-27828fba3973 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:12.438752 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42fc28d-1120-40f7-98a8-27828fba3973 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:12.439173 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42fc28d-1120-40f7-98a8-27828fba3973 None admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:12.443873 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42fc28d-1120-40f7-98a8-27828fba3973 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:12.443873 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42fc28d-1120-40f7-98a8-27828fba3973 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:12.443873 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42fc28d-1120-40f7-98a8-27828fba3973 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:12.465140 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 426/863] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Aug 10 08:24:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:12.467405 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42fc28d-1120-40f7-98a8-27828fba3973 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:12.472966 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 438/864] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:24:12 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 08:24:12.498189 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacd6402-9c7e-465b-9b2f-60e10e5dc44f None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:12.534462 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:12.606043 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42fc28d-1120-40f7-98a8-27828fba3973 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:12.632615 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacd6402-9c7e-465b-9b2f-60e10e5dc44f None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:12.645418 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a71369-acb7-47d0-84e6-de2a64e2ab44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:12.646007 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a71369-acb7-47d0-84e6-de2a64e2ab44 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:12.646248 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a71369-acb7-47d0-84e6-de2a64e2ab44 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:12.646626 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a71369-acb7-47d0-84e6-de2a64e2ab44 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:12.655386 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a71369-acb7-47d0-84e6-de2a64e2ab44 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:12.656805 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a71369-acb7-47d0-84e6-de2a64e2ab44 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:12.669153 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196ffb29-48fa-4cb3-96ee-c2872d3fd0eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:12.669764 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196ffb29-48fa-4cb3-96ee-c2872d3fd0eb None admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:12.669955 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-53a71369-acb7-47d0-84e6-de2a64e2ab44 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 08:24:12.669955 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:12.669955 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:24:12.669955 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:24:12.669955 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:24:12.669955 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:24:12.669955 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:24:12.669955 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:24:12.669955 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:24:12.669955 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:24:12.669955 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:24:12.669955 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:24:12.669955 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 08:24:12.669955 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 08:24:12.669955 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 08:24:12.670855 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 08:24:12.670855 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:24:12.670855 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:24:12.670855 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:24:12.670855 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:24:12.670855 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:24:12.670855 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:24:12.670855 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:24:12.670855 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:24:12.670855 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:24:12.670855 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:24:12.670855 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:24:12.670855 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:24:12.670855 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:24:12.670855 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:24:12.670855 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:24:12.670855 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:24:12.671774 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 08:24:12.671774 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 08:24:12.671774 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 08:24:12.671774 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 08:24:12.671774 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 08:24:12.671774 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 08:24:12.671774 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 08:24:12.671774 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:24:12.672215 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196ffb29-48fa-4cb3-96ee-c2872d3fd0eb None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:12.672215 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196ffb29-48fa-4cb3-96ee-c2872d3fd0eb None admin] PATH_INFO: `/v3/services` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:12.672215 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196ffb29-48fa-4cb3-96ee-c2872d3fd0eb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:12.672397 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196ffb29-48fa-4cb3-96ee-c2872d3fd0eb None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:12.675277 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 427/865] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Aug 10 08:24:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:24:12.676805 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4be9ca0-6791-4e0f-90ae-d174e20290c6 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:12.677133 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4be9ca0-6791-4e0f-90ae-d174e20290c6 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:12.677435 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4be9ca0-6791-4e0f-90ae-d174e20290c6 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:12.678723 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 428/866] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:24:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:24:12.679410 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4be9ca0-6791-4e0f-90ae-d174e20290c6 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:12.755592 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 439/867] 10.222.0.57 () {60 vars in 1141 bytes} [Wed Aug 10 08:24:12 2022] POST /identity/v3/services => generated 251 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 08:24:12.756975 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190f0c57-9fd8-4d85-944b-7f10fb75608a None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:12.757263 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190f0c57-9fd8-4d85-944b-7f10fb75608a None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:12.757531 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190f0c57-9fd8-4d85-944b-7f10fb75608a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:12.784125 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-190f0c57-9fd8-4d85-944b-7f10fb75608a None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:24:12.786276 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-190f0c57-9fd8-4d85-944b-7f10fb75608a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:24:12.807356 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:12.817432 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4be9ca0-6791-4e0f-90ae-d174e20290c6 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:12.839835 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:12.870332 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0601b79e-6c5c-4439-9233-dbb4ec9bab63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:12.871379 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0601b79e-6c5c-4439-9233-dbb4ec9bab63 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:12.871660 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0601b79e-6c5c-4439-9233-dbb4ec9bab63 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:12.871981 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0601b79e-6c5c-4439-9233-dbb4ec9bab63 None admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:12.872988 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0601b79e-6c5c-4439-9233-dbb4ec9bab63 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:12.873303 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0601b79e-6c5c-4439-9233-dbb4ec9bab63 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:12.874967 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0601b79e-6c5c-4439-9233-dbb4ec9bab63 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:12.881039 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 440/868] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:24:12 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 08:24:12.902320 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 429/869] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Aug 10 08:24:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:12.909266 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190f0c57-9fd8-4d85-944b-7f10fb75608a None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:12.910611 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0601b79e-6c5c-4439-9233-dbb4ec9bab63 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:13.044703 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190f0c57-9fd8-4d85-944b-7f10fb75608a None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:13.050614 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0601b79e-6c5c-4439-9233-dbb4ec9bab63 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:13.088111 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce043595-148b-4a7f-bd7c-15fca4374b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:13.089686 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce043595-148b-4a7f-bd7c-15fca4374b27 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:13.090230 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce043595-148b-4a7f-bd7c-15fca4374b27 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:13.090648 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce043595-148b-4a7f-bd7c-15fca4374b27 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:13.091529 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce043595-148b-4a7f-bd7c-15fca4374b27 None admin] RBAC: Adding query filter params (role.id=ec3751692d9c4ecf81b4a71c936c46be, scope.project.id=b49e381419e940278bcb6111899589b5, user.id=b4b26ba3a9e04c38b52db2a46198a53c) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:13.091979 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce043595-148b-4a7f-bd7c-15fca4374b27 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:13.093820 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd671df-d18e-4738-9f5f-168904a9536e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:13.094038 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce043595-148b-4a7f-bd7c-15fca4374b27 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:13.094665 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd671df-d18e-4738-9f5f-168904a9536e None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:13.095044 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd671df-d18e-4738-9f5f-168904a9536e None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:13.095417 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd671df-d18e-4738-9f5f-168904a9536e None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:13.097397 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd671df-d18e-4738-9f5f-168904a9536e None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:13.099393 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd671df-d18e-4738-9f5f-168904a9536e None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:13.102489 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 441/870] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Aug 10 08:24:12 2022] GET /identity/v3/role_assignments?user.id=b4b26ba3a9e04c38b52db2a46198a53c&scope.project.id=b49e381419e940278bcb6111899589b5&role.id=ec3751692d9c4ecf81b4a71c936c46be => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:13.112949 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-edd671df-d18e-4738-9f5f-168904a9536e None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 08:24:13.112949 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:13.112949 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 08:24:13.112949 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 08:24:13.112949 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 08:24:13.112949 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 08:24:13.112949 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 08:24:13.112949 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 08:24:13.112949 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 08:24:13.112949 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:24:13.112949 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 08:24:13.112949 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:24:13.112949 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:13.112949 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:24:13.112949 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:24:13.112949 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:24:13.114156 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:24:13.114156 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:24:13.114156 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:24:13.114156 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:24:13.114156 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:24:13.114156 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:24:13.114156 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:24:13.114156 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 08:24:13.114156 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 08:24:13.114156 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 08:24:13.114156 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 08:24:13.114156 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:24:13.114156 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:24:13.114156 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:24:13.114156 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:24:13.114156 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:24:13.114156 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:24:13.115293 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:24:13.115293 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:24:13.115293 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:24:13.115293 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:24:13.115293 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:24:13.115293 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:24:13.115293 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:24:13.115293 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:24:13.115293 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:24:13.115293 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:24:13.115293 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 08:24:13.115293 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 08:24:13.115293 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 08:24:13.115293 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:24:13.115293 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 430/871] 10.222.0.57 () {56 vars in 1132 bytes} [Wed Aug 10 08:24:12 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 08:24:13.117892 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce043595-148b-4a7f-bd7c-15fca4374b27 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:13.166213 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:13.243290 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce043595-148b-4a7f-bd7c-15fca4374b27 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:13.282189 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f311e3f-d661-40db-9839-de0d90a35f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:13.282810 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f311e3f-d661-40db-9839-de0d90a35f94 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:13.283092 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f311e3f-d661-40db-9839-de0d90a35f94 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:13.284066 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f311e3f-d661-40db-9839-de0d90a35f94 None admin] PATH_INFO: `/v3/services` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:13.284898 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f311e3f-d661-40db-9839-de0d90a35f94 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:13.285284 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f311e3f-d661-40db-9839-de0d90a35f94 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:13.286692 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f311e3f-d661-40db-9839-de0d90a35f94 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:13.292091 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 442/872] 10.222.0.57 () {56 vars in 1126 bytes} [Wed Aug 10 08:24:13 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:24:13.297527 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd671df-d18e-4738-9f5f-168904a9536e None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:13.332652 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:13.423180 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd671df-d18e-4738-9f5f-168904a9536e None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:13.459789 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36c2bf4-4d1c-42d8-a2cf-82329c0e6a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:13.460765 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36c2bf4-4d1c-42d8-a2cf-82329c0e6a06 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:13.461278 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36c2bf4-4d1c-42d8-a2cf-82329c0e6a06 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:13.461741 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36c2bf4-4d1c-42d8-a2cf-82329c0e6a06 None admin] PATH_INFO: `/v3/services` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:13.462558 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36c2bf4-4d1c-42d8-a2cf-82329c0e6a06 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:13.463064 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36c2bf4-4d1c-42d8-a2cf-82329c0e6a06 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:13.464628 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36c2bf4-4d1c-42d8-a2cf-82329c0e6a06 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:13.469819 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 431/873] 10.222.0.57 () {56 vars in 1126 bytes} [Wed Aug 10 08:24:13 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:24:14.491110 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72988a8e-cc9c-4421-8b72-2904da64009b None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:14.491627 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72988a8e-cc9c-4421-8b72-2904da64009b None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:14.491627 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72988a8e-cc9c-4421-8b72-2904da64009b None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:14.493289 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 443/874] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:24:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:24:14.498658 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33cf6dd-fd7d-4758-b7a2-402fc67d31c8 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:14.499459 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33cf6dd-fd7d-4758-b7a2-402fc67d31c8 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:14.499930 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33cf6dd-fd7d-4758-b7a2-402fc67d31c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:14.519709 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-d33cf6dd-fd7d-4758-b7a2-402fc67d31c8 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:24:14.521301 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-d33cf6dd-fd7d-4758-b7a2-402fc67d31c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:24:14.590928 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 432/875] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:24:14 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 08:24:14.614293 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72988a8e-cc9c-4421-8b72-2904da64009b None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:14.718584 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:14.722916 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72988a8e-cc9c-4421-8b72-2904da64009b None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:14.757797 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f78aed-1ed0-4749-bb00-e73b82fce6f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:14.758818 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f78aed-1ed0-4749-bb00-e73b82fce6f2 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:14.759293 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f78aed-1ed0-4749-bb00-e73b82fce6f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:14.759754 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f78aed-1ed0-4749-bb00-e73b82fce6f2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:14.770401 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f78aed-1ed0-4749-bb00-e73b82fce6f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 08:24:14.775274 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-48f78aed-1ed0-4749-bb00-e73b82fce6f2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 08:24:14.775274 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:14.775274 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:24:14.775274 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:24:14.775274 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:24:14.775274 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:24:14.775274 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:24:14.775274 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:24:14.775274 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:24:14.775274 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:24:14.775274 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:24:14.775274 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:24:14.775274 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 08:24:14.775274 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 08:24:14.775274 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 08:24:14.775274 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 08:24:14.776199 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 08:24:14.776199 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 08:24:14.776199 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 08:24:14.776199 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 08:24:14.776199 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:24:14.776199 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:24:14.776199 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:24:14.776199 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:24:14.776199 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:24:14.776199 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:24:14.776199 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:24:14.776199 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:24:14.776199 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:24:14.776199 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:24:14.776199 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:24:14.776199 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:24:14.777094 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:24:14.777094 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:24:14.777094 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:24:14.777094 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:24:14.777094 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 08:24:14.777094 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 08:24:14.777094 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 08:24:14.777094 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 08:24:14.777094 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 08:24:14.777094 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 08:24:14.777094 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 08:24:14.777094 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:24:14.779732 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 444/876] 10.222.0.57 () {56 vars in 1120 bytes} [Wed Aug 10 08:24:14 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:24:14.786871 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33cf6dd-fd7d-4758-b7a2-402fc67d31c8 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:14.806816 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:14.891346 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33cf6dd-fd7d-4758-b7a2-402fc67d31c8 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:14.927454 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdf4fe5-bac8-4d16-80b3-6bf665e923fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:14.928427 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdf4fe5-bac8-4d16-80b3-6bf665e923fa None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:14.928900 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdf4fe5-bac8-4d16-80b3-6bf665e923fa None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:14.929410 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdf4fe5-bac8-4d16-80b3-6bf665e923fa None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:14.930634 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdf4fe5-bac8-4d16-80b3-6bf665e923fa None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:14.931134 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdf4fe5-bac8-4d16-80b3-6bf665e923fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:14.932764 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdf4fe5-bac8-4d16-80b3-6bf665e923fa None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:14.938977 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f172d3-a99d-47b7-9031-dfeb23aa651f None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:14.939258 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f172d3-a99d-47b7-9031-dfeb23aa651f None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:14.939827 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f172d3-a99d-47b7-9031-dfeb23aa651f None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:14.941467 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 445/877] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:24:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:24:14.942719 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 433/878] 10.222.0.57 () {56 vars in 1115 bytes} [Wed Aug 10 08:24:14 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:14.946412 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd432db-7753-4fdc-86bd-06930c553300 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:14.946661 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd432db-7753-4fdc-86bd-06930c553300 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:14.946940 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd432db-7753-4fdc-86bd-06930c553300 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:14.948845 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdf4fe5-bac8-4d16-80b3-6bf665e923fa None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:14.965783 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-5cd432db-7753-4fdc-86bd-06930c553300 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:24:14.966971 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-5cd432db-7753-4fdc-86bd-06930c553300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:24:15.037395 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:15.045154 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 446/879] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:24:14 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 08:24:15.057474 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdf4fe5-bac8-4d16-80b3-6bf665e923fa None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:15.061002 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd432db-7753-4fdc-86bd-06930c553300 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:15.095021 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f159d8-203d-4aaa-b01e-59e95a9d394c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:15.096691 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f159d8-203d-4aaa-b01e-59e95a9d394c None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:15.097296 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f159d8-203d-4aaa-b01e-59e95a9d394c None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:15.097843 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f159d8-203d-4aaa-b01e-59e95a9d394c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:15.109519 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f159d8-203d-4aaa-b01e-59e95a9d394c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:15.111238 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f159d8-203d-4aaa-b01e-59e95a9d394c None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:15.128527 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-f3f159d8-203d-4aaa-b01e-59e95a9d394c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:24:15.128527 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:15.128527 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:24:15.128527 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:24:15.128527 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:24:15.128527 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:24:15.128527 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:24:15.128527 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:24:15.128527 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:24:15.128527 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:24:15.128527 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:24:15.128527 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:24:15.128527 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 08:24:15.128527 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 08:24:15.128527 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 08:24:15.128527 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 08:24:15.129568 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:24:15.129568 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:24:15.129568 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:24:15.129568 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:24:15.129568 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:24:15.129568 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:24:15.129568 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:24:15.129568 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:24:15.129568 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:24:15.129568 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:24:15.129568 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:24:15.129568 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:24:15.129568 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:24:15.129568 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:24:15.129568 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:24:15.129568 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:24:15.129568 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 08:24:15.130468 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 08:24:15.130468 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:24:15.130468 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:24:15.137003 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 434/880] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:24:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 08:24:15.138646 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f159d8-203d-4aaa-b01e-59e95a9d394c None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:15.156408 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:15.166968 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd432db-7753-4fdc-86bd-06930c553300 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:15.203147 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e754c1aa-dd8d-44f8-ac5b-1a50cf38a694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:15.203658 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e754c1aa-dd8d-44f8-ac5b-1a50cf38a694 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:15.203936 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e754c1aa-dd8d-44f8-ac5b-1a50cf38a694 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:15.204210 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e754c1aa-dd8d-44f8-ac5b-1a50cf38a694 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:15.204879 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e754c1aa-dd8d-44f8-ac5b-1a50cf38a694 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:15.207015 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e754c1aa-dd8d-44f8-ac5b-1a50cf38a694 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:15.216017 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-e754c1aa-dd8d-44f8-ac5b-1a50cf38a694 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 08:24:15.216017 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:15.216017 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 08:24:15.216017 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 08:24:15.216017 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 08:24:15.216017 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 08:24:15.216017 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 08:24:15.216017 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 08:24:15.216017 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 08:24:15.216017 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:24:15.216017 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 08:24:15.216017 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:24:15.216017 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:15.216017 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:24:15.216017 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:24:15.216017 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:24:15.216017 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:24:15.217094 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:24:15.217094 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:24:15.217094 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:24:15.217094 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:24:15.217094 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:24:15.217094 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:24:15.217094 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 08:24:15.217094 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 08:24:15.217094 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 08:24:15.217094 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 08:24:15.217094 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:24:15.217094 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:24:15.217094 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:24:15.217094 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:24:15.217094 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:24:15.217094 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:24:15.217952 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:24:15.217952 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:24:15.217952 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:24:15.217952 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:24:15.217952 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:24:15.217952 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:24:15.217952 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:24:15.217952 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:24:15.217952 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:24:15.217952 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:24:15.217952 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 08:24:15.217952 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 08:24:15.217952 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 08:24:15.217952 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:24:15.222422 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 447/881] 10.222.0.57 () {56 vars in 1114 bytes} [Wed Aug 10 08:24:15 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 08:24:15.223326 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e754c1aa-dd8d-44f8-ac5b-1a50cf38a694 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:15.253036 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f159d8-203d-4aaa-b01e-59e95a9d394c None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:15.283164 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:15.288492 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f32566c-63f8-4e7c-9b01-4bef87884199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:15.289071 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f32566c-63f8-4e7c-9b01-4bef87884199 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:15.289347 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f32566c-63f8-4e7c-9b01-4bef87884199 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:15.289622 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f32566c-63f8-4e7c-9b01-4bef87884199 None admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:15.290377 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f32566c-63f8-4e7c-9b01-4bef87884199 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:15.290683 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f32566c-63f8-4e7c-9b01-4bef87884199 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:15.292178 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f32566c-63f8-4e7c-9b01-4bef87884199 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:15.304442 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 435/882] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Aug 10 08:24:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:24:15.304442 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6929207-5d76-4923-b5d6-2d34adafe5ce None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:15.304442 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6929207-5d76-4923-b5d6-2d34adafe5ce None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:15.304442 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6929207-5d76-4923-b5d6-2d34adafe5ce None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:15.304442 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 436/883] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:24:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:24:15.310011 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6929207-5d76-4923-b5d6-2d34adafe5ce None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:15.332235 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e754c1aa-dd8d-44f8-ac5b-1a50cf38a694 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:15.366386 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885a8d96-8da8-4fac-a8f6-98c2e65d9320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:15.366989 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885a8d96-8da8-4fac-a8f6-98c2e65d9320 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:15.367314 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885a8d96-8da8-4fac-a8f6-98c2e65d9320 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:15.367545 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885a8d96-8da8-4fac-a8f6-98c2e65d9320 None admin] PATH_INFO: `/v3/services` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:15.368144 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885a8d96-8da8-4fac-a8f6-98c2e65d9320 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:15.368449 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885a8d96-8da8-4fac-a8f6-98c2e65d9320 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:15.370097 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885a8d96-8da8-4fac-a8f6-98c2e65d9320 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:15.374686 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 448/884] 10.222.0.57 () {56 vars in 1114 bytes} [Wed Aug 10 08:24:15 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:15.375817 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1703a96e-ce77-48e7-9180-766a1206ec8e None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:15.376097 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1703a96e-ce77-48e7-9180-766a1206ec8e None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:15.376369 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1703a96e-ce77-48e7-9180-766a1206ec8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:15.399542 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-1703a96e-ce77-48e7-9180-766a1206ec8e None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:24:15.400871 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-1703a96e-ce77-48e7-9180-766a1206ec8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:24:15.418783 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:15.434199 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6929207-5d76-4923-b5d6-2d34adafe5ce None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:15.473491 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1a74ea-781a-4ad1-bbfe-ffa8c18a96b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:15.474117 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1a74ea-781a-4ad1-bbfe-ffa8c18a96b2 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:15.474405 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1a74ea-781a-4ad1-bbfe-ffa8c18a96b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:15.474671 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1a74ea-781a-4ad1-bbfe-ffa8c18a96b2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:15.482696 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1a74ea-781a-4ad1-bbfe-ffa8c18a96b2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:15.484128 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1a74ea-781a-4ad1-bbfe-ffa8c18a96b2 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:15.486832 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 449/885] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:24:15 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 08:24:15.487721 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1703a96e-ce77-48e7-9180-766a1206ec8e None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:15.512081 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-0d1a74ea-781a-4ad1-bbfe-ffa8c18a96b2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 08:24:15.512081 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:15.512081 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:24:15.512081 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:24:15.512081 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:24:15.512081 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:24:15.512081 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:24:15.512081 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:24:15.512081 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:24:15.512081 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:24:15.512081 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:24:15.512081 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:24:15.512081 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 08:24:15.512081 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 08:24:15.512081 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 08:24:15.512081 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 08:24:15.513477 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:24:15.513477 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:24:15.513477 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 08:24:15.513477 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 08:24:15.513477 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 08:24:15.513477 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 08:24:15.513477 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:24:15.513477 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:24:15.513477 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:24:15.513477 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:24:15.513477 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:24:15.513477 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:24:15.513477 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:24:15.513477 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:24:15.513477 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:24:15.513477 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:24:15.513477 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:24:15.513477 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:24:15.514435 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:24:15.514435 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:24:15.514435 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 08:24:15.514435 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 08:24:15.514435 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 08:24:15.514435 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 08:24:15.514435 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 08:24:15.514435 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 08:24:15.514435 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 08:24:15.514435 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:24:15.514435 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 437/886] 10.222.0.57 () {56 vars in 1117 bytes} [Wed Aug 10 08:24:15 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:24:15.515536 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1a74ea-781a-4ad1-bbfe-ffa8c18a96b2 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:15.576320 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:15.590703 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1703a96e-ce77-48e7-9180-766a1206ec8e None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:15.623099 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1a74ea-781a-4ad1-bbfe-ffa8c18a96b2 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:15.624486 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a39107-d7de-4534-89ee-b6b7b3af973f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:15.625110 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a39107-d7de-4534-89ee-b6b7b3af973f None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:15.625389 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a39107-d7de-4534-89ee-b6b7b3af973f None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:15.625654 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a39107-d7de-4534-89ee-b6b7b3af973f None admin] PATH_INFO: `/v3/services` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:15.626269 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a39107-d7de-4534-89ee-b6b7b3af973f None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:15.626568 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a39107-d7de-4534-89ee-b6b7b3af973f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:15.628997 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a39107-d7de-4534-89ee-b6b7b3af973f None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:15.633641 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 450/887] 10.222.0.57 () {56 vars in 1114 bytes} [Wed Aug 10 08:24:15 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:15.634425 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a39107-d7de-4534-89ee-b6b7b3af973f None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:15.648340 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:15.660278 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69dd5af-fe9c-47c9-967a-a9aa02d85145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:15.660910 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69dd5af-fe9c-47c9-967a-a9aa02d85145 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:15.661218 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69dd5af-fe9c-47c9-967a-a9aa02d85145 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:15.661487 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69dd5af-fe9c-47c9-967a-a9aa02d85145 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:15.662070 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69dd5af-fe9c-47c9-967a-a9aa02d85145 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:15.663426 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69dd5af-fe9c-47c9-967a-a9aa02d85145 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:15.671860 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-b69dd5af-fe9c-47c9-967a-a9aa02d85145 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 08:24:15.671860 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:15.671860 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 08:24:15.671860 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 08:24:15.671860 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 08:24:15.671860 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 08:24:15.671860 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 08:24:15.671860 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 08:24:15.671860 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 08:24:15.671860 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:24:15.671860 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 08:24:15.671860 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:24:15.671860 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:15.671860 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:24:15.671860 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:24:15.671860 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:24:15.671860 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:24:15.673069 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:24:15.673069 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:24:15.673069 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:24:15.673069 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:24:15.673069 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:24:15.673069 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:24:15.673069 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 08:24:15.673069 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 08:24:15.673069 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 08:24:15.673069 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 08:24:15.673069 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:24:15.673069 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:24:15.673069 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:24:15.673069 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:24:15.673069 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:24:15.673069 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:24:15.674677 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:24:15.674677 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:24:15.674677 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:24:15.674677 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:24:15.674677 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:24:15.674677 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:24:15.674677 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:24:15.674677 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:24:15.674677 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:24:15.674677 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:24:15.674677 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 08:24:15.674677 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 08:24:15.674677 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 08:24:15.674677 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:24:15.674677 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 438/888] 10.222.0.57 () {56 vars in 1126 bytes} [Wed Aug 10 08:24:15 2022] GET /identity/v3/services/object-store => generated 93 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:24:15.675489 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d26c884-f1b4-4d19-9721-5237131c5137 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:15.675489 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d26c884-f1b4-4d19-9721-5237131c5137 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:15.675489 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d26c884-f1b4-4d19-9721-5237131c5137 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:15.676593 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 439/889] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:24:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:24:15.677325 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d26c884-f1b4-4d19-9721-5237131c5137 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:15.737229 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a39107-d7de-4534-89ee-b6b7b3af973f None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:15.772108 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4babd4e6-7ad3-42ca-9c6e-3b8b166fc358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:15.772710 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4babd4e6-7ad3-42ca-9c6e-3b8b166fc358 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:15.773014 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4babd4e6-7ad3-42ca-9c6e-3b8b166fc358 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:15.773288 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4babd4e6-7ad3-42ca-9c6e-3b8b166fc358 None admin] PATH_INFO: `/v3/users` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:15.774005 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4babd4e6-7ad3-42ca-9c6e-3b8b166fc358 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:15.774315 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4babd4e6-7ad3-42ca-9c6e-3b8b166fc358 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:15.775743 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4babd4e6-7ad3-42ca-9c6e-3b8b166fc358 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:15.784467 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d26c884-f1b4-4d19-9721-5237131c5137 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:15.792714 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 451/890] 10.222.0.57 () {56 vars in 1149 bytes} [Wed Aug 10 08:24:15 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:15.794020 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d1e402-1f81-4537-9335-b99bd5669388 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:15.794309 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d1e402-1f81-4537-9335-b99bd5669388 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:15.794831 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d1e402-1f81-4537-9335-b99bd5669388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:15.801538 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:15.812423 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-91d1e402-1f81-4537-9335-b99bd5669388 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:24:15.813646 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-91d1e402-1f81-4537-9335-b99bd5669388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:24:15.820204 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aac7e7-de49-4027-8ee5-113994d85bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:15.820802 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aac7e7-de49-4027-8ee5-113994d85bb0 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:15.821124 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aac7e7-de49-4027-8ee5-113994d85bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:15.821395 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aac7e7-de49-4027-8ee5-113994d85bb0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:15.822110 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3aac7e7-de49-4027-8ee5-113994d85bb0 None admin] RBAC: Adding query filter params (interface=public, service_id=111c165c14304148a8b605123cc6faf2, region_id=RegionOne) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:15.822393 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3aac7e7-de49-4027-8ee5-113994d85bb0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:15.823748 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3aac7e7-de49-4027-8ee5-113994d85bb0 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:15.831767 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 440/891] 10.222.0.57 () {56 vars in 1251 bytes} [Wed Aug 10 08:24:15 2022] GET /identity/v3/endpoints?service_id=111c165c14304148a8b605123cc6faf2&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:15.832565 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aac7e7-de49-4027-8ee5-113994d85bb0 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:15.879625 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:15.884094 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 452/892] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:24:15 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 08:24:15.884992 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d1e402-1f81-4537-9335-b99bd5669388 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:15.935611 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aac7e7-de49-4027-8ee5-113994d85bb0 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:15.970012 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222b37af-603e-4d07-9172-1fc8c6e23246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:15.970615 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222b37af-603e-4d07-9172-1fc8c6e23246 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:15.971411 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222b37af-603e-4d07-9172-1fc8c6e23246 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:15.971640 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222b37af-603e-4d07-9172-1fc8c6e23246 None admin] PATH_INFO: `/v3/services` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:15.972414 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222b37af-603e-4d07-9172-1fc8c6e23246 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:15.972720 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222b37af-603e-4d07-9172-1fc8c6e23246 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:15.974177 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222b37af-603e-4d07-9172-1fc8c6e23246 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:15.978607 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 441/893] 10.222.0.57 () {56 vars in 1122 bytes} [Wed Aug 10 08:24:15 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:15.979374 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222b37af-603e-4d07-9172-1fc8c6e23246 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:15.996935 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d1e402-1f81-4537-9335-b99bd5669388 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:16.025962 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:16.031876 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e3c305-200a-4f5f-b997-4b69c0f515a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:16.032479 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e3c305-200a-4f5f-b997-4b69c0f515a4 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:16.032792 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e3c305-200a-4f5f-b997-4b69c0f515a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:16.033092 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e3c305-200a-4f5f-b997-4b69c0f515a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:16.079277 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222b37af-603e-4d07-9172-1fc8c6e23246 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:16.099160 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:16.101211 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e3c305-200a-4f5f-b997-4b69c0f515a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:16.102710 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e3c305-200a-4f5f-b997-4b69c0f515a4 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:16.115638 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc5adf8-5fa3-413a-ac98-f22d27d89c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:16.116242 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc5adf8-5fa3-413a-ac98-f22d27d89c78 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:16.116518 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc5adf8-5fa3-413a-ac98-f22d27d89c78 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:16.116798 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc5adf8-5fa3-413a-ac98-f22d27d89c78 None admin] PATH_INFO: `/v3/services` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:16.117423 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc5adf8-5fa3-413a-ac98-f22d27d89c78 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:16.118804 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc5adf8-5fa3-413a-ac98-f22d27d89c78 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:16.124542 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 442/894] 10.222.0.57 () {56 vars in 1087 bytes} [Wed Aug 10 08:24:15 2022] GET /identity/v3/services => generated 1011 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:24:16.125327 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc5adf8-5fa3-413a-ac98-f22d27d89c78 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:16.214412 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 453/895] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Aug 10 08:24:15 2022] GET /identity/v3/auth/tokens => generated 1262 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 08:24:16.215529 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e3c305-200a-4f5f-b997-4b69c0f515a4 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:16.264660 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc5adf8-5fa3-413a-ac98-f22d27d89c78 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:16.275610 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:16.305247 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c14824-93ad-4716-ba18-b6665f006357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:16.305861 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c14824-93ad-4716-ba18-b6665f006357 None admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:16.306149 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c14824-93ad-4716-ba18-b6665f006357 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:16.307673 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c14824-93ad-4716-ba18-b6665f006357 None admin] PATH_INFO: `/v3/services` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:16.307673 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c14824-93ad-4716-ba18-b6665f006357 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:16.308733 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c14824-93ad-4716-ba18-b6665f006357 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:16.327242 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:16.330200 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e3c305-200a-4f5f-b997-4b69c0f515a4 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:16.363965 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3581b41-741e-4f2c-83bc-911bf54f5521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:16.364491 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3581b41-741e-4f2c-83bc-911bf54f5521 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:16.364770 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3581b41-741e-4f2c-83bc-911bf54f5521 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:16.365076 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3581b41-741e-4f2c-83bc-911bf54f5521 None admin] PATH_INFO: `/v3/services` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:16.365707 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3581b41-741e-4f2c-83bc-911bf54f5521 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:16.366007 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3581b41-741e-4f2c-83bc-911bf54f5521 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:16.367387 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3581b41-741e-4f2c-83bc-911bf54f5521 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:16.376985 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 454/896] 10.222.0.57 () {56 vars in 1122 bytes} [Wed Aug 10 08:24:16 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:16.376985 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3581b41-741e-4f2c-83bc-911bf54f5521 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:16.414439 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 443/897] 10.222.0.57 () {60 vars in 1141 bytes} [Wed Aug 10 08:24:16 2022] POST /identity/v3/services => generated 269 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:24:16.525979 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3581b41-741e-4f2c-83bc-911bf54f5521 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:16.572343 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7c8b04-30df-4f6f-8810-b4800a2ef3eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:16.573022 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7c8b04-30df-4f6f-8810-b4800a2ef3eb None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:16.573308 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7c8b04-30df-4f6f-8810-b4800a2ef3eb None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:16.573579 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7c8b04-30df-4f6f-8810-b4800a2ef3eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:16.585688 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7c8b04-30df-4f6f-8810-b4800a2ef3eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:16.587337 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7c8b04-30df-4f6f-8810-b4800a2ef3eb None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:16.604484 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-5d7c8b04-30df-4f6f-8810-b4800a2ef3eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:24:16.604484 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:16.604484 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:24:16.604484 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:24:16.604484 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:24:16.604484 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:24:16.604484 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:24:16.604484 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:24:16.604484 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:24:16.604484 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:24:16.604484 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:24:16.604484 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:24:16.604484 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 08:24:16.604484 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 08:24:16.604484 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 08:24:16.604484 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 08:24:16.606107 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:24:16.606107 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:24:16.606107 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:24:16.606107 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:24:16.606107 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:24:16.606107 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:24:16.606107 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:24:16.606107 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:24:16.606107 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:24:16.606107 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:24:16.606107 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:24:16.606107 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:24:16.606107 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:24:16.606107 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:24:16.606107 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:24:16.606107 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:24:16.606107 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 08:24:16.607022 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 08:24:16.607022 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:24:16.607022 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:24:16.607022 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 455/898] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:24:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:24:16.612835 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c14824-93ad-4716-ba18-b6665f006357 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:16.724545 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c14824-93ad-4716-ba18-b6665f006357 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:16.760480 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69c8972-9cce-4cce-85b3-622851fbdb5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:16.761371 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69c8972-9cce-4cce-85b3-622851fbdb5b None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:16.761800 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69c8972-9cce-4cce-85b3-622851fbdb5b None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:16.762221 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69c8972-9cce-4cce-85b3-622851fbdb5b None admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:16.763039 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69c8972-9cce-4cce-85b3-622851fbdb5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:16.763494 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69c8972-9cce-4cce-85b3-622851fbdb5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:16.765063 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69c8972-9cce-4cce-85b3-622851fbdb5b None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:16.778852 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 444/899] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Aug 10 08:24:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:24:16.786015 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7c8b04-30df-4f6f-8810-b4800a2ef3eb None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:16.803958 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:16.894615 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7c8b04-30df-4f6f-8810-b4800a2ef3eb None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:16.931159 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869ef3c9-2389-4acb-97ee-39fd5e33767b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:16.932143 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869ef3c9-2389-4acb-97ee-39fd5e33767b None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:16.932419 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869ef3c9-2389-4acb-97ee-39fd5e33767b None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:16.932684 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869ef3c9-2389-4acb-97ee-39fd5e33767b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:16.939677 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869ef3c9-2389-4acb-97ee-39fd5e33767b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:16.941250 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869ef3c9-2389-4acb-97ee-39fd5e33767b None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:16.952654 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-869ef3c9-2389-4acb-97ee-39fd5e33767b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 08:24:16.952654 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:16.952654 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:24:16.952654 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:24:16.952654 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:24:16.952654 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:24:16.952654 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:24:16.952654 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:24:16.952654 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:24:16.952654 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:24:16.952654 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:24:16.952654 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:24:16.952654 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 08:24:16.952654 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 08:24:16.952654 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 08:24:16.953601 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 08:24:16.953601 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:24:16.953601 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:24:16.953601 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:24:16.953601 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:24:16.953601 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:24:16.953601 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:24:16.953601 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:24:16.953601 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:24:16.953601 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:24:16.953601 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:24:16.953601 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:24:16.953601 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:24:16.953601 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:24:16.953601 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:24:16.953601 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:24:16.953601 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:24:16.954503 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 08:24:16.954503 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 08:24:16.954503 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 08:24:16.954503 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 08:24:16.954503 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 08:24:16.954503 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 08:24:16.954503 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 08:24:16.954503 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:24:16.954503 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 456/900] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Aug 10 08:24:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:24:16.961339 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69c8972-9cce-4cce-85b3-622851fbdb5b None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:16.992446 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:17.070443 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69c8972-9cce-4cce-85b3-622851fbdb5b None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:17.106394 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d4d21a-bcdc-438a-bfb7-e20874b920cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:17.107346 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d4d21a-bcdc-438a-bfb7-e20874b920cd None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:17.107932 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d4d21a-bcdc-438a-bfb7-e20874b920cd None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:17.108431 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d4d21a-bcdc-438a-bfb7-e20874b920cd None admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:17.109409 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d4d21a-bcdc-438a-bfb7-e20874b920cd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:17.109933 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d4d21a-bcdc-438a-bfb7-e20874b920cd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:17.112146 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d4d21a-bcdc-438a-bfb7-e20874b920cd None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:17.124876 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 445/901] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Aug 10 08:24:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:17.132553 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869ef3c9-2389-4acb-97ee-39fd5e33767b None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:17.212143 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:17.244321 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869ef3c9-2389-4acb-97ee-39fd5e33767b None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:17.281458 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cec11f-1c9c-49c7-b602-2e64c8475770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:17.282461 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cec11f-1c9c-49c7-b602-2e64c8475770 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:17.282946 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cec11f-1c9c-49c7-b602-2e64c8475770 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:17.283505 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cec11f-1c9c-49c7-b602-2e64c8475770 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:17.284723 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cec11f-1c9c-49c7-b602-2e64c8475770 None admin] RBAC: Adding query filter params (role.id=c30407dbef394307b4d8d3dc9ca4bcaa, scope.project.id=b49e381419e940278bcb6111899589b5, user.id=c140d31bd2b84c48a170e3ca90a041ce) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:17.285355 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cec11f-1c9c-49c7-b602-2e64c8475770 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:17.287138 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cec11f-1c9c-49c7-b602-2e64c8475770 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:17.294397 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 457/902] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Aug 10 08:24:17 2022] GET /identity/v3/role_assignments?user.id=c140d31bd2b84c48a170e3ca90a041ce&scope.project.id=b49e381419e940278bcb6111899589b5&role.id=c30407dbef394307b4d8d3dc9ca4bcaa => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:18.222682 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25f208b-1abe-48c2-806a-cde302340feb None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:18.223247 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25f208b-1abe-48c2-806a-cde302340feb None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:18.223247 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25f208b-1abe-48c2-806a-cde302340feb None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:18.224857 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 446/903] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:24:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 08:24:18.230100 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48202843-5420-4eb8-83c9-ad1906b5a864 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:18.230512 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48202843-5420-4eb8-83c9-ad1906b5a864 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:18.230699 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48202843-5420-4eb8-83c9-ad1906b5a864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:18.249557 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-48202843-5420-4eb8-83c9-ad1906b5a864 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:24:18.250812 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-48202843-5420-4eb8-83c9-ad1906b5a864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:24:18.326837 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 458/904] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:24:18 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 08:24:18.346664 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25f208b-1abe-48c2-806a-cde302340feb None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:18.457719 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:18.467293 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25f208b-1abe-48c2-806a-cde302340feb None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:18.504181 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badc6c68-08a3-49ec-a456-7b31b5add1ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:18.504736 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badc6c68-08a3-49ec-a456-7b31b5add1ca None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:18.505039 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badc6c68-08a3-49ec-a456-7b31b5add1ca None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:18.505309 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badc6c68-08a3-49ec-a456-7b31b5add1ca None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:18.505895 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badc6c68-08a3-49ec-a456-7b31b5add1ca None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:18.507319 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badc6c68-08a3-49ec-a456-7b31b5add1ca None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:18.516343 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-badc6c68-08a3-49ec-a456-7b31b5add1ca None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 08:24:18.516343 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:18.516343 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 08:24:18.516343 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 08:24:18.516343 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 08:24:18.516343 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 08:24:18.516343 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 08:24:18.516343 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 08:24:18.516343 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 08:24:18.516343 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:24:18.516343 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 08:24:18.516343 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:24:18.516343 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:18.516343 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:24:18.516343 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:24:18.516343 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:24:18.516343 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:24:18.517389 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:24:18.517389 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:24:18.517389 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:24:18.517389 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:24:18.517389 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:24:18.517389 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:24:18.517389 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 08:24:18.517389 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 08:24:18.517389 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 08:24:18.517389 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 08:24:18.517389 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:24:18.517389 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:24:18.517389 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:24:18.517389 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:24:18.517389 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:24:18.517389 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:24:18.518369 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:24:18.518369 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:24:18.518369 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:24:18.518369 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:24:18.518369 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:24:18.518369 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:24:18.518369 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:24:18.518369 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:24:18.518369 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:24:18.518369 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:24:18.518369 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 08:24:18.518369 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 08:24:18.518369 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 08:24:18.518369 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:24:18.518369 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 447/905] 10.222.0.57 () {56 vars in 1114 bytes} [Wed Aug 10 08:24:18 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:24:18.524108 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48202843-5420-4eb8-83c9-ad1906b5a864 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:18.541512 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3189d57e-90db-47be-a8fe-e7fbfdcb55e6 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:18.541836 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3189d57e-90db-47be-a8fe-e7fbfdcb55e6 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:18.542056 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3189d57e-90db-47be-a8fe-e7fbfdcb55e6 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:18.543626 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 448/906] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:24:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:24:18.548334 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2b5fca-181d-470b-981e-9e1165378af8 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:18.548608 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2b5fca-181d-470b-981e-9e1165378af8 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:18.550068 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2b5fca-181d-470b-981e-9e1165378af8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:18.568472 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-ca2b5fca-181d-470b-981e-9e1165378af8 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:24:18.569676 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-ca2b5fca-181d-470b-981e-9e1165378af8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:24:18.595171 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:18.632380 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48202843-5420-4eb8-83c9-ad1906b5a864 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:18.652591 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 449/907] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:24:18 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 08:24:18.654198 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041d6e55-5c78-4e17-980d-a92724d68b63 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:18.654673 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041d6e55-5c78-4e17-980d-a92724d68b63 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:18.655527 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041d6e55-5c78-4e17-980d-a92724d68b63 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:18.657651 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 450/908] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:24:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:24:18.663098 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca110cec-c0f7-4d68-9fd7-c88325745659 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:18.663381 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca110cec-c0f7-4d68-9fd7-c88325745659 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:18.663658 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca110cec-c0f7-4d68-9fd7-c88325745659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:18.676049 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff086f6-01f0-44d4-94b2-465cefb4521e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:18.677044 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff086f6-01f0-44d4-94b2-465cefb4521e None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:18.677535 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff086f6-01f0-44d4-94b2-465cefb4521e None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:18.677877 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff086f6-01f0-44d4-94b2-465cefb4521e None admin] PATH_INFO: `/v3/services` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:18.678521 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff086f6-01f0-44d4-94b2-465cefb4521e None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:18.678827 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff086f6-01f0-44d4-94b2-465cefb4521e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:18.680198 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff086f6-01f0-44d4-94b2-465cefb4521e None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:18.681382 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-ca110cec-c0f7-4d68-9fd7-c88325745659 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:24:18.682567 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-ca110cec-c0f7-4d68-9fd7-c88325745659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:24:18.685187 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 459/909] 10.222.0.57 () {56 vars in 1114 bytes} [Wed Aug 10 08:24:18 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:18.685884 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff086f6-01f0-44d4-94b2-465cefb4521e None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:18.761161 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 451/910] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:24:18 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 08:24:18.761161 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca110cec-c0f7-4d68-9fd7-c88325745659 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:18.780369 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:18.804010 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff086f6-01f0-44d4-94b2-465cefb4521e None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:18.836641 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:18.841886 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb73e99-d60b-412a-a221-5e3199fbfc1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:18.842484 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb73e99-d60b-412a-a221-5e3199fbfc1d None admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:18.842771 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb73e99-d60b-412a-a221-5e3199fbfc1d None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:18.843038 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb73e99-d60b-412a-a221-5e3199fbfc1d None admin] PATH_INFO: `/v3/services` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:18.843673 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb73e99-d60b-412a-a221-5e3199fbfc1d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:18.845208 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb73e99-d60b-412a-a221-5e3199fbfc1d None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:18.873214 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca110cec-c0f7-4d68-9fd7-c88325745659 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:18.908811 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f325cd2-5fc6-4d6b-b7b9-6fe830af6ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:18.909614 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f325cd2-5fc6-4d6b-b7b9-6fe830af6ca2 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:18.909899 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f325cd2-5fc6-4d6b-b7b9-6fe830af6ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:18.910168 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f325cd2-5fc6-4d6b-b7b9-6fe830af6ca2 None admin] PATH_INFO: `/v3/services` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:18.910800 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f325cd2-5fc6-4d6b-b7b9-6fe830af6ca2 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:18.911101 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f325cd2-5fc6-4d6b-b7b9-6fe830af6ca2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:18.912463 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f325cd2-5fc6-4d6b-b7b9-6fe830af6ca2 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:18.917301 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 452/911] 10.222.0.57 () {56 vars in 1114 bytes} [Wed Aug 10 08:24:18 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:18.918085 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f325cd2-5fc6-4d6b-b7b9-6fe830af6ca2 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:18.918947 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 460/912] 10.222.0.57 () {60 vars in 1141 bytes} [Wed Aug 10 08:24:18 2022] POST /identity/v3/services => generated 241 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:24:18.922992 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb73e99-d60b-412a-a221-5e3199fbfc1d None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:19.022746 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:19.049339 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f325cd2-5fc6-4d6b-b7b9-6fe830af6ca2 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:19.059979 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb73e99-d60b-412a-a221-5e3199fbfc1d None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:19.089693 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666d5894-0c25-4caf-bfc8-71ef1564fe07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:19.090489 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666d5894-0c25-4caf-bfc8-71ef1564fe07 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:19.090865 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666d5894-0c25-4caf-bfc8-71ef1564fe07 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:19.091343 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666d5894-0c25-4caf-bfc8-71ef1564fe07 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:19.092030 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666d5894-0c25-4caf-bfc8-71ef1564fe07 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:19.093531 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666d5894-0c25-4caf-bfc8-71ef1564fe07 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:19.101457 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3bbb6b-8843-465e-a3a5-385ca62bde52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:19.102100 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3bbb6b-8843-465e-a3a5-385ca62bde52 None admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:19.102379 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3bbb6b-8843-465e-a3a5-385ca62bde52 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:19.102649 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3bbb6b-8843-465e-a3a5-385ca62bde52 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:19.103362 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3bbb6b-8843-465e-a3a5-385ca62bde52 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:19.104758 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3bbb6b-8843-465e-a3a5-385ca62bde52 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:19.104858 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-666d5894-0c25-4caf-bfc8-71ef1564fe07 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 08:24:19.104858 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:19.104858 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 08:24:19.104858 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 08:24:19.104858 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 08:24:19.104858 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 08:24:19.104858 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 08:24:19.104858 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 08:24:19.104858 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 08:24:19.104858 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:24:19.104858 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 08:24:19.104858 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:24:19.104858 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:19.104858 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:24:19.104858 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:24:19.104858 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:24:19.105870 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:24:19.105870 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:24:19.105870 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:24:19.105870 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:24:19.105870 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:24:19.105870 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:24:19.105870 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:24:19.105870 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 08:24:19.105870 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 08:24:19.105870 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 08:24:19.105870 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 08:24:19.105870 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:24:19.105870 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:24:19.105870 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:24:19.105870 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:24:19.105870 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:24:19.105870 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:24:19.106821 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:24:19.106821 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:24:19.106821 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:24:19.106821 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:24:19.106821 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:24:19.106821 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:24:19.106821 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:24:19.106821 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:24:19.106821 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:24:19.106821 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:24:19.106821 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 08:24:19.106821 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 08:24:19.106821 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 08:24:19.106821 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:24:19.107581 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 453/913] 10.222.0.57 () {56 vars in 1132 bytes} [Wed Aug 10 08:24:18 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:24:19.113706 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666d5894-0c25-4caf-bfc8-71ef1564fe07 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:19.211296 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 461/914] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 08:24:18 2022] POST /identity/v3/endpoints => generated 346 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:24:19.230212 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666d5894-0c25-4caf-bfc8-71ef1564fe07 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:19.280569 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322212c2-c69c-4596-ae41-5f0cce30bd0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:19.281527 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322212c2-c69c-4596-ae41-5f0cce30bd0d None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:19.281997 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322212c2-c69c-4596-ae41-5f0cce30bd0d None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:19.282412 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322212c2-c69c-4596-ae41-5f0cce30bd0d None admin] PATH_INFO: `/v3/services` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:19.284422 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322212c2-c69c-4596-ae41-5f0cce30bd0d None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:19.284908 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322212c2-c69c-4596-ae41-5f0cce30bd0d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:19.286569 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322212c2-c69c-4596-ae41-5f0cce30bd0d None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:19.289054 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:19.294273 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 454/915] 10.222.0.57 () {56 vars in 1126 bytes} [Wed Aug 10 08:24:19 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:24:19.300341 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3bbb6b-8843-465e-a3a5-385ca62bde52 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:19.352973 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:19.402095 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0524f7-651a-406d-8dbd-de82018c2c36 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:19.402404 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0524f7-651a-406d-8dbd-de82018c2c36 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:19.402633 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0524f7-651a-406d-8dbd-de82018c2c36 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:19.403971 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 455/916] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:24:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:24:19.410154 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f774260e-d221-4de7-bcc3-c474dc482f73 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:19.415668 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f774260e-d221-4de7-bcc3-c474dc482f73 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:19.415668 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f774260e-d221-4de7-bcc3-c474dc482f73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:19.418897 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3bbb6b-8843-465e-a3a5-385ca62bde52 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:19.433104 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-f774260e-d221-4de7-bcc3-c474dc482f73 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:24:19.434295 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-f774260e-d221-4de7-bcc3-c474dc482f73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:24:19.469495 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43751093-e94c-4031-bff7-17c6699c4528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:19.470219 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43751093-e94c-4031-bff7-17c6699c4528 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:19.470520 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43751093-e94c-4031-bff7-17c6699c4528 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:19.472196 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43751093-e94c-4031-bff7-17c6699c4528 None admin] PATH_INFO: `/v3/services` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:19.472843 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43751093-e94c-4031-bff7-17c6699c4528 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:19.473298 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43751093-e94c-4031-bff7-17c6699c4528 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:19.474676 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43751093-e94c-4031-bff7-17c6699c4528 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:19.481247 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 462/917] 10.222.0.57 () {56 vars in 1126 bytes} [Wed Aug 10 08:24:19 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:19.487044 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43751093-e94c-4031-bff7-17c6699c4528 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:19.528442 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 456/918] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:24:19 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 08:24:19.551775 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f774260e-d221-4de7-bcc3-c474dc482f73 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:19.622543 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43751093-e94c-4031-bff7-17c6699c4528 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:19.633462 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:19.666214 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f774260e-d221-4de7-bcc3-c474dc482f73 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:19.669613 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e07b8d7-4e58-4635-8e6b-599b56e85f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:19.673765 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e07b8d7-4e58-4635-8e6b-599b56e85f42 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:19.674788 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e07b8d7-4e58-4635-8e6b-599b56e85f42 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:19.675862 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e07b8d7-4e58-4635-8e6b-599b56e85f42 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:19.676783 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e07b8d7-4e58-4635-8e6b-599b56e85f42 None admin] RBAC: Adding query filter params (interface=public, service_id=51e08eb3816e4256a3c844f41a98bb6b, region_id=RegionOne) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:19.677321 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e07b8d7-4e58-4635-8e6b-599b56e85f42 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:19.679808 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e07b8d7-4e58-4635-8e6b-599b56e85f42 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:19.687507 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 463/919] 10.222.0.57 () {56 vars in 1251 bytes} [Wed Aug 10 08:24:19 2022] GET /identity/v3/endpoints?service_id=51e08eb3816e4256a3c844f41a98bb6b&interface=public®ion_id=RegionOne => generated 194 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:19.693408 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e07b8d7-4e58-4635-8e6b-599b56e85f42 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:19.698730 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:19.704900 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c507e613-9546-48f1-aa66-0b531f89ee5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:19.706167 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c507e613-9546-48f1-aa66-0b531f89ee5f None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:19.706545 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c507e613-9546-48f1-aa66-0b531f89ee5f None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:19.706879 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c507e613-9546-48f1-aa66-0b531f89ee5f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:19.719488 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c507e613-9546-48f1-aa66-0b531f89ee5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 08:24:19.725468 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-c507e613-9546-48f1-aa66-0b531f89ee5f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 08:24:19.725468 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:19.725468 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:24:19.725468 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:24:19.725468 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:24:19.725468 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:24:19.725468 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:24:19.725468 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:24:19.725468 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:24:19.725468 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:24:19.725468 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:24:19.725468 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:24:19.725468 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 08:24:19.725468 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 08:24:19.725468 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 08:24:19.725468 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 08:24:19.726466 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 08:24:19.726466 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 08:24:19.726466 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 08:24:19.726466 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 08:24:19.726466 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:24:19.726466 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:24:19.726466 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:24:19.726466 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:24:19.726466 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:24:19.726466 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:24:19.726466 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:24:19.726466 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:24:19.726466 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:24:19.726466 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:24:19.726466 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:24:19.726466 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:24:19.727369 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:24:19.727369 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:24:19.727369 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:24:19.727369 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:24:19.727369 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 08:24:19.727369 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 08:24:19.727369 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 08:24:19.727369 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 08:24:19.727369 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 08:24:19.727369 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 08:24:19.727369 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 08:24:19.727369 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:24:19.729646 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 457/920] 10.222.0.57 () {56 vars in 1120 bytes} [Wed Aug 10 08:24:19 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 08:24:19.737676 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c507e613-9546-48f1-aa66-0b531f89ee5f None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:19.811068 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e07b8d7-4e58-4635-8e6b-599b56e85f42 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:19.846823 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c507e613-9546-48f1-aa66-0b531f89ee5f None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:19.848367 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc434ee8-37d0-4d63-b135-a9df4b98ca10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:19.849328 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc434ee8-37d0-4d63-b135-a9df4b98ca10 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:19.849875 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc434ee8-37d0-4d63-b135-a9df4b98ca10 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:19.850339 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc434ee8-37d0-4d63-b135-a9df4b98ca10 None admin] PATH_INFO: `/v3/services` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:19.851175 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc434ee8-37d0-4d63-b135-a9df4b98ca10 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:19.852723 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc434ee8-37d0-4d63-b135-a9df4b98ca10 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:19.859275 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 464/921] 10.222.0.57 () {56 vars in 1087 bytes} [Wed Aug 10 08:24:19 2022] GET /identity/v3/services => generated 1497 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 08:24:19.884278 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2677a5a6-6752-40fd-89fa-a9de0c2f7b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:19.884907 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2677a5a6-6752-40fd-89fa-a9de0c2f7b19 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:19.885213 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2677a5a6-6752-40fd-89fa-a9de0c2f7b19 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:19.885481 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2677a5a6-6752-40fd-89fa-a9de0c2f7b19 None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:19.886166 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2677a5a6-6752-40fd-89fa-a9de0c2f7b19 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:19.886465 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2677a5a6-6752-40fd-89fa-a9de0c2f7b19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:19.887865 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2677a5a6-6752-40fd-89fa-a9de0c2f7b19 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:19.898301 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 458/922] 10.222.0.57 () {56 vars in 1115 bytes} [Wed Aug 10 08:24:19 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:19.903980 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc434ee8-37d0-4d63-b135-a9df4b98ca10 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:19.957330 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:20.025448 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc434ee8-37d0-4d63-b135-a9df4b98ca10 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:20.060734 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f7e502-8c5e-4351-809c-fd57f3b1d49a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:20.061459 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f7e502-8c5e-4351-809c-fd57f3b1d49a None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:20.061714 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f7e502-8c5e-4351-809c-fd57f3b1d49a None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:20.062036 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f7e502-8c5e-4351-809c-fd57f3b1d49a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:20.073772 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f7e502-8c5e-4351-809c-fd57f3b1d49a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:20.075288 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f7e502-8c5e-4351-809c-fd57f3b1d49a None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:20.089453 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-d8f7e502-8c5e-4351-809c-fd57f3b1d49a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:24:20.089453 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:20.089453 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:24:20.089453 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:24:20.089453 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:24:20.089453 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:24:20.089453 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:24:20.089453 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:24:20.089453 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:24:20.089453 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:24:20.089453 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:24:20.089453 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:24:20.089453 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 08:24:20.089453 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 08:24:20.089453 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 08:24:20.089453 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 08:24:20.090531 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:24:20.090531 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:24:20.090531 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:24:20.090531 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:24:20.090531 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:24:20.090531 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:24:20.090531 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:24:20.090531 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:24:20.090531 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:24:20.090531 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:24:20.090531 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:24:20.090531 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:24:20.090531 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:24:20.090531 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:24:20.090531 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:24:20.090531 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:24:20.090531 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 08:24:20.091412 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 08:24:20.091412 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:24:20.091412 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:24:20.091412 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 465/923] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:24:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:24:20.096999 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2677a5a6-6752-40fd-89fa-a9de0c2f7b19 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:20.152827 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:20.204555 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2677a5a6-6752-40fd-89fa-a9de0c2f7b19 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:20.243198 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201ce0d1-4618-424b-ad4d-c376ce97b351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:20.243884 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201ce0d1-4618-424b-ad4d-c376ce97b351 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:20.244107 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201ce0d1-4618-424b-ad4d-c376ce97b351 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:20.244582 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201ce0d1-4618-424b-ad4d-c376ce97b351 None admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:20.245326 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201ce0d1-4618-424b-ad4d-c376ce97b351 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:20.245626 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201ce0d1-4618-424b-ad4d-c376ce97b351 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:20.247000 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201ce0d1-4618-424b-ad4d-c376ce97b351 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:20.260385 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 459/924] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Aug 10 08:24:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:24:20.266153 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f7e502-8c5e-4351-809c-fd57f3b1d49a None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:20.356810 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:20.373596 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f7e502-8c5e-4351-809c-fd57f3b1d49a None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:20.413543 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddb84dd-6a8a-4938-9e88-92700d108cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:20.414174 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddb84dd-6a8a-4938-9e88-92700d108cc9 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:20.414452 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddb84dd-6a8a-4938-9e88-92700d108cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:20.414719 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddb84dd-6a8a-4938-9e88-92700d108cc9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:20.422570 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddb84dd-6a8a-4938-9e88-92700d108cc9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:20.424779 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddb84dd-6a8a-4938-9e88-92700d108cc9 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:20.435813 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-1ddb84dd-6a8a-4938-9e88-92700d108cc9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 08:24:20.435813 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:20.435813 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:24:20.435813 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:24:20.435813 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:24:20.435813 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:24:20.435813 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:24:20.435813 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:24:20.435813 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:24:20.435813 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:24:20.435813 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:24:20.435813 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:24:20.435813 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 08:24:20.435813 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 08:24:20.435813 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 08:24:20.435813 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 08:24:20.436846 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:24:20.436846 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:24:20.436846 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 08:24:20.436846 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 08:24:20.436846 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 08:24:20.436846 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 08:24:20.436846 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:24:20.436846 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:24:20.436846 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:24:20.436846 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:24:20.436846 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:24:20.436846 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:24:20.436846 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:24:20.436846 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:24:20.436846 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:24:20.436846 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:24:20.436846 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:24:20.436846 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:24:20.437832 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:24:20.437832 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:24:20.437832 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 08:24:20.437832 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 08:24:20.437832 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 08:24:20.437832 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 08:24:20.437832 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 08:24:20.437832 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 08:24:20.437832 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 08:24:20.437832 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:24:20.437832 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 466/925] 10.222.0.57 () {56 vars in 1117 bytes} [Wed Aug 10 08:24:20 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:24:20.443369 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201ce0d1-4618-424b-ad4d-c376ce97b351 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:20.552559 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201ce0d1-4618-424b-ad4d-c376ce97b351 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:20.567099 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:20.589891 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf1e757-2067-4616-8c73-6b118e15f042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:20.590499 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf1e757-2067-4616-8c73-6b118e15f042 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:20.590778 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf1e757-2067-4616-8c73-6b118e15f042 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:20.591048 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf1e757-2067-4616-8c73-6b118e15f042 None admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:20.591780 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf1e757-2067-4616-8c73-6b118e15f042 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:20.592082 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf1e757-2067-4616-8c73-6b118e15f042 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:20.593573 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf1e757-2067-4616-8c73-6b118e15f042 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:20.612824 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 460/926] 10.222.0.57 () {56 vars in 1149 bytes} [Wed Aug 10 08:24:20 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:20.618374 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddb84dd-6a8a-4938-9e88-92700d108cc9 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:20.724044 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddb84dd-6a8a-4938-9e88-92700d108cc9 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:20.761303 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1764a5d-5b38-4981-8d6c-abd65f8d055a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:20.761933 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1764a5d-5b38-4981-8d6c-abd65f8d055a None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:20.762214 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1764a5d-5b38-4981-8d6c-abd65f8d055a None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:20.762485 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1764a5d-5b38-4981-8d6c-abd65f8d055a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:20.774062 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1764a5d-5b38-4981-8d6c-abd65f8d055a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:20.775575 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1764a5d-5b38-4981-8d6c-abd65f8d055a None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:20.781968 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:20.790962 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-b1764a5d-5b38-4981-8d6c-abd65f8d055a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:24:20.790962 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:20.790962 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:24:20.790962 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:24:20.790962 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:24:20.790962 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:24:20.790962 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:24:20.790962 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:24:20.790962 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:24:20.790962 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:24:20.790962 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:24:20.790962 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:24:20.790962 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 08:24:20.790962 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 08:24:20.790962 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 08:24:20.790962 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 08:24:20.795743 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:24:20.795743 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:24:20.795743 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:24:20.795743 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:24:20.795743 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:24:20.795743 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:24:20.795743 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:24:20.795743 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:24:20.795743 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:24:20.795743 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:24:20.795743 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:24:20.795743 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:24:20.795743 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:24:20.795743 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:24:20.795743 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:24:20.795743 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:24:20.795743 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 08:24:20.796881 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 08:24:20.796881 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:24:20.796881 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:24:20.796881 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 467/927] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:24:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:24:20.801480 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf1e757-2067-4616-8c73-6b118e15f042 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:20.909762 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf1e757-2067-4616-8c73-6b118e15f042 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:20.946037 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efbe46f-254b-48c9-bbea-bdae6eba8bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:20.946914 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efbe46f-254b-48c9-bbea-bdae6eba8bbf None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:20.947342 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efbe46f-254b-48c9-bbea-bdae6eba8bbf None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:20.947755 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efbe46f-254b-48c9-bbea-bdae6eba8bbf None admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:20.948558 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efbe46f-254b-48c9-bbea-bdae6eba8bbf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:20.949038 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efbe46f-254b-48c9-bbea-bdae6eba8bbf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:20.950678 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efbe46f-254b-48c9-bbea-bdae6eba8bbf None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:20.963736 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 461/928] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Aug 10 08:24:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:20.969476 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1764a5d-5b38-4981-8d6c-abd65f8d055a None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:21.079451 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1764a5d-5b38-4981-8d6c-abd65f8d055a None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:21.116821 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0400ce-552e-4946-a262-487acd7ddae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:21.117759 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0400ce-552e-4946-a262-487acd7ddae2 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:21.118189 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0400ce-552e-4946-a262-487acd7ddae2 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:21.118663 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0400ce-552e-4946-a262-487acd7ddae2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:21.120495 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f407dc-d638-4821-8f99-ad59e846968a None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:21.120961 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f407dc-d638-4821-8f99-ad59e846968a None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:21.121462 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f407dc-d638-4821-8f99-ad59e846968a None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:21.123572 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 462/929] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:24:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:24:21.130174 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0400ce-552e-4946-a262-487acd7ddae2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:21.130960 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eb0f92-7309-433a-b223-06c28955f8df None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:21.131615 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eb0f92-7309-433a-b223-06c28955f8df None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:21.131745 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0400ce-552e-4946-a262-487acd7ddae2 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:21.132212 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eb0f92-7309-433a-b223-06c28955f8df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:21.145660 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-1b0400ce-552e-4946-a262-487acd7ddae2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 08:24:21.145660 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:21.145660 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:24:21.145660 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:24:21.145660 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:24:21.145660 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:24:21.145660 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:24:21.145660 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:24:21.145660 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:24:21.145660 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:24:21.145660 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:24:21.145660 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:24:21.145660 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 08:24:21.145660 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 08:24:21.145660 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 08:24:21.146791 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 08:24:21.146791 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:24:21.146791 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:24:21.146791 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:24:21.146791 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:24:21.146791 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:24:21.146791 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:24:21.146791 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:24:21.146791 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:24:21.146791 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:24:21.146791 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:24:21.146791 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:24:21.146791 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:24:21.146791 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:24:21.146791 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:24:21.146791 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:24:21.146791 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:24:21.154023 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 08:24:21.154023 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 08:24:21.154023 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 08:24:21.154023 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 08:24:21.154023 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 08:24:21.154023 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 08:24:21.154023 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 08:24:21.154023 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:24:21.154023 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 468/930] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Aug 10 08:24:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:24:21.158664 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0400ce-552e-4946-a262-487acd7ddae2 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:21.166457 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-56eb0f92-7309-433a-b223-06c28955f8df None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:24:21.168031 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-56eb0f92-7309-433a-b223-06c28955f8df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:24:21.195181 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:21.241684 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:21.246856 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 463/931] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:24:21 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 08:24:21.273551 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eb0f92-7309-433a-b223-06c28955f8df None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:21.295986 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0400ce-552e-4946-a262-487acd7ddae2 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:21.332059 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aadec09-3517-45e4-9425-4eb437ae6976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:21.332815 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aadec09-3517-45e4-9425-4eb437ae6976 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:21.333341 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aadec09-3517-45e4-9425-4eb437ae6976 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:21.333813 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aadec09-3517-45e4-9425-4eb437ae6976 None admin] PATH_INFO: `/v3/projects` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:21.335018 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aadec09-3517-45e4-9425-4eb437ae6976 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:21.335524 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aadec09-3517-45e4-9425-4eb437ae6976 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:21.337605 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aadec09-3517-45e4-9425-4eb437ae6976 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:21.350863 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 469/932] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Aug 10 08:24:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:21.356715 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aadec09-3517-45e4-9425-4eb437ae6976 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:21.398562 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eb0f92-7309-433a-b223-06c28955f8df None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:21.444367 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc32423-4800-49e8-84aa-fd5044f67814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:21.445571 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc32423-4800-49e8-84aa-fd5044f67814 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:21.445853 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc32423-4800-49e8-84aa-fd5044f67814 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:21.446126 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc32423-4800-49e8-84aa-fd5044f67814 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:21.446998 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc32423-4800-49e8-84aa-fd5044f67814 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:21.449548 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc32423-4800-49e8-84aa-fd5044f67814 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:21.461249 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-3cc32423-4800-49e8-84aa-fd5044f67814 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 08:24:21.461249 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:21.461249 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 08:24:21.461249 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 08:24:21.461249 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 08:24:21.461249 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 08:24:21.461249 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 08:24:21.461249 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 08:24:21.461249 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 08:24:21.461249 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:24:21.461249 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 08:24:21.461249 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:24:21.461249 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:21.461249 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:24:21.461249 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:24:21.461249 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:24:21.461249 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:24:21.463278 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:24:21.463278 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:24:21.463278 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:24:21.463278 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:24:21.463278 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:24:21.463278 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:24:21.463278 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 08:24:21.463278 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 08:24:21.463278 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 08:24:21.463278 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 08:24:21.463278 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:24:21.463278 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:24:21.463278 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:24:21.463278 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:24:21.463278 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:24:21.463278 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:24:21.464965 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:24:21.464965 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:24:21.464965 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:24:21.464965 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:24:21.464965 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:24:21.464965 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:24:21.464965 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:24:21.464965 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:24:21.464965 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:24:21.464965 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:24:21.464965 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 08:24:21.464965 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 08:24:21.464965 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 08:24:21.464965 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:24:21.464965 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 464/933] 10.222.0.57 () {56 vars in 1126 bytes} [Wed Aug 10 08:24:21 2022] GET /identity/v3/services/object-store => generated 93 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 08:24:21.470529 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc32423-4800-49e8-84aa-fd5044f67814 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:21.491781 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aadec09-3517-45e4-9425-4eb437ae6976 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:21.510394 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:21.512718 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:21.530017 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fb05ed-ea0e-4523-8764-2c9089c1cc51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:21.530677 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fb05ed-ea0e-4523-8764-2c9089c1cc51 None admin] REQUEST_METHOD: `PUT` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:21.531004 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fb05ed-ea0e-4523-8764-2c9089c1cc51 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:21.531317 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fb05ed-ea0e-4523-8764-2c9089c1cc51 None admin] PATH_INFO: `/v3/projects/b49e381419e940278bcb6111899589b5/users/c140d31bd2b84c48a170e3ca90a041ce/roles/c30407dbef394307b4d8d3dc9ca4bcaa` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:21.533888 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fb05ed-ea0e-4523-8764-2c9089c1cc51 None admin] RBAC: Authorizing `identity:create_grant(project_id=b49e381419e940278bcb6111899589b5, user_id=c140d31bd2b84c48a170e3ca90a041ce, role_id=c30407dbef394307b4d8d3dc9ca4bcaa)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:21.535340 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fb05ed-ea0e-4523-8764-2c9089c1cc51 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:21.594410 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc32423-4800-49e8-84aa-fd5044f67814 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:21.629421 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 470/934] 10.222.0.57 () {58 vars in 1439 bytes} [Wed Aug 10 08:24:21 2022] PUT /identity/v3/projects/b49e381419e940278bcb6111899589b5/users/c140d31bd2b84c48a170e3ca90a041ce/roles/c30407dbef394307b4d8d3dc9ca4bcaa => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:24:21.631242 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c739c4f-6fd3-499a-a198-f9afd5c2a6d2 None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:21.631679 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c739c4f-6fd3-499a-a198-f9afd5c2a6d2 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:21.632133 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c739c4f-6fd3-499a-a198-f9afd5c2a6d2 None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:21.633738 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 471/935] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:24:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:24:21.639452 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3c8254-faa6-44f8-9b78-8907fb426e31 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:21.645194 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8c3bc2-857c-4e1d-ab89-4cb5b17aa9c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:21.645446 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3c8254-faa6-44f8-9b78-8907fb426e31 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:21.645654 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3c8254-faa6-44f8-9b78-8907fb426e31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:21.645962 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8c3bc2-857c-4e1d-ab89-4cb5b17aa9c0 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:21.646303 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8c3bc2-857c-4e1d-ab89-4cb5b17aa9c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:21.646681 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8c3bc2-857c-4e1d-ab89-4cb5b17aa9c0 None admin] PATH_INFO: `/v3/services` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:21.647381 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8c3bc2-857c-4e1d-ab89-4cb5b17aa9c0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:21.647746 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8c3bc2-857c-4e1d-ab89-4cb5b17aa9c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:21.649224 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8c3bc2-857c-4e1d-ab89-4cb5b17aa9c0 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:21.654866 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 465/936] 10.222.0.57 () {56 vars in 1122 bytes} [Wed Aug 10 08:24:21 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:24:21.661173 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8c3bc2-857c-4e1d-ab89-4cb5b17aa9c0 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:21.682764 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-9b3c8254-faa6-44f8-9b78-8907fb426e31 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:24:21.684363 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-9b3c8254-faa6-44f8-9b78-8907fb426e31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:24:21.787179 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 472/937] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:24:21 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 08:24:21.788807 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:21.804293 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8c3bc2-857c-4e1d-ab89-4cb5b17aa9c0 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:21.812124 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3c8254-faa6-44f8-9b78-8907fb426e31 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:21.846835 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e8645e-f4ec-4d5e-b599-bd8a141f446e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:21.847573 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e8645e-f4ec-4d5e-b599-bd8a141f446e None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:21.847835 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e8645e-f4ec-4d5e-b599-bd8a141f446e None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:21.848105 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e8645e-f4ec-4d5e-b599-bd8a141f446e None admin] PATH_INFO: `/v3/services` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:21.848736 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e8645e-f4ec-4d5e-b599-bd8a141f446e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:21.850094 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e8645e-f4ec-4d5e-b599-bd8a141f446e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:21.851511 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e8645e-f4ec-4d5e-b599-bd8a141f446e None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:21.860956 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 466/938] 10.222.0.57 () {56 vars in 1122 bytes} [Wed Aug 10 08:24:21 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:24:21.863629 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e8645e-f4ec-4d5e-b599-bd8a141f446e None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:21.937314 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3c8254-faa6-44f8-9b78-8907fb426e31 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:21.947893 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:21.978334 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a4ed3e-8e1f-40b0-9039-302467425e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:21.978996 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a4ed3e-8e1f-40b0-9039-302467425e6f None admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:21.979275 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a4ed3e-8e1f-40b0-9039-302467425e6f None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:21.979537 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a4ed3e-8e1f-40b0-9039-302467425e6f None admin] PATH_INFO: `/v3/projects` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:21.980179 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a4ed3e-8e1f-40b0-9039-302467425e6f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:21.981643 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a4ed3e-8e1f-40b0-9039-302467425e6f None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:21.984251 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: WARNING keystone.server.flask.common [None req-31a4ed3e-8e1f-40b0-9039-302467425e6f None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 10 08:24:22.000348 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e8645e-f4ec-4d5e-b599-bd8a141f446e None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:22.029006 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.sql.core [None req-31a4ed3e-8e1f-40b0-9039-302467425e6f None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 10 08:24:22.029006 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 10 08:24:22.029006 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [parameters: {'id': '43070d3d2c65450f9daa7cbf11207731', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 10 08:24:22.029006 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80461) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 10 08:24:22.037386 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f62a0f2-94c6-4e7f-af8c-b7aa706c4b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:22.037921 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f62a0f2-94c6-4e7f-af8c-b7aa706c4b12 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:22.038197 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f62a0f2-94c6-4e7f-af8c-b7aa706c4b12 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:22.038464 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f62a0f2-94c6-4e7f-af8c-b7aa706c4b12 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:22.039163 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f62a0f2-94c6-4e7f-af8c-b7aa706c4b12 None admin] RBAC: Adding query filter params (interface=public, service_id=db11dbab82b34ba69856d999b49c60c7, region_id=RegionOne) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:22.039467 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f62a0f2-94c6-4e7f-af8c-b7aa706c4b12 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:22.040808 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f62a0f2-94c6-4e7f-af8c-b7aa706c4b12 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:22.051011 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 467/939] 10.222.0.57 () {56 vars in 1251 bytes} [Wed Aug 10 08:24:21 2022] GET /identity/v3/endpoints?service_id=db11dbab82b34ba69856d999b49c60c7&interface=public®ion_id=RegionOne => generated 194 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:22.051210 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-31a4ed3e-8e1f-40b0-9039-302467425e6f None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 43070d3d2c65450f9daa7cbf11207731.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 43070d3d2c65450f9daa7cbf11207731. Aug 10 08:24:22.051210 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:22.051210 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 10 08:24:22.051210 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 10 08:24:22.051210 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 10 08:24:22.051210 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 10 08:24:22.051210 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 10 08:24:22.051210 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application result = self._query(query) Aug 10 08:24:22.051210 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 10 08:24:22.051210 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application conn.query(q) Aug 10 08:24:22.051210 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 10 08:24:22.051210 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 10 08:24:22.051210 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 10 08:24:22.053616 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application result.read() Aug 10 08:24:22.053616 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 10 08:24:22.053616 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 10 08:24:22.053616 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 10 08:24:22.053616 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 10 08:24:22.053616 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 10 08:24:22.053616 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 10 08:24:22.053616 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 10 08:24:22.053616 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 10 08:24:22.053616 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 10 08:24:22.053616 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:24:22.053616 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 10 08:24:22.053616 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:24:22.053616 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:22.053616 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 10 08:24:22.053616 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 10 08:24:22.053616 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 10 08:24:22.053616 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 10 08:24:22.054896 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 10 08:24:22.054896 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application next(self.gen) Aug 10 08:24:22.054896 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 10 08:24:22.054896 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application yield resource Aug 10 08:24:22.054896 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 10 08:24:22.054896 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application next(self.gen) Aug 10 08:24:22.054896 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 10 08:24:22.054896 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application self.session.rollback() Aug 10 08:24:22.054896 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 10 08:24:22.054896 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application self.force_reraise() Aug 10 08:24:22.054896 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 10 08:24:22.054896 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise self.value Aug 10 08:24:22.054896 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 10 08:24:22.054896 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 10 08:24:22.054896 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 10 08:24:22.054896 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application session.commit() Aug 10 08:24:22.054896 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 10 08:24:22.058510 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 10 08:24:22.058510 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 10 08:24:22.058510 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application self._prepare_impl() Aug 10 08:24:22.058510 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 10 08:24:22.058510 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application self.session.flush() Aug 10 08:24:22.058510 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 10 08:24:22.058510 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application self._flush(objects) Aug 10 08:24:22.058510 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 10 08:24:22.058510 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 10 08:24:22.058510 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 10 08:24:22.058510 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application compat.raise_( Aug 10 08:24:22.058510 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 10 08:24:22.058510 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception Aug 10 08:24:22.058510 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 10 08:24:22.058510 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application flush_context.execute() Aug 10 08:24:22.058510 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 10 08:24:22.058510 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rec.execute(self) Aug 10 08:24:22.061458 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 10 08:24:22.061458 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 10 08:24:22.061458 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 10 08:24:22.061458 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 10 08:24:22.061458 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 10 08:24:22.061458 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 10 08:24:22.061458 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 10 08:24:22.061458 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 10 08:24:22.061458 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 10 08:24:22.061458 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 10 08:24:22.061458 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 10 08:24:22.061458 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 10 08:24:22.061458 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 10 08:24:22.061458 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 10 08:24:22.061458 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 10 08:24:22.062481 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 10 08:24:22.062481 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 10 08:24:22.062481 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception Aug 10 08:24:22.062481 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 10 08:24:22.062481 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 10 08:24:22.062481 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 10 08:24:22.062481 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 10 08:24:22.062481 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 10 08:24:22.062481 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application result = self._query(query) Aug 10 08:24:22.062481 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 10 08:24:22.062481 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application conn.query(q) Aug 10 08:24:22.062481 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 10 08:24:22.062481 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 10 08:24:22.062481 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 10 08:24:22.062481 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application result.read() Aug 10 08:24:22.062481 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 10 08:24:22.062481 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 10 08:24:22.063618 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 10 08:24:22.063618 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 10 08:24:22.063618 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 10 08:24:22.063618 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 10 08:24:22.063618 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 10 08:24:22.063618 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 10 08:24:22.063618 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 10 08:24:22.063618 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 10 08:24:22.063618 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [parameters: {'id': '43070d3d2c65450f9daa7cbf11207731', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 10 08:24:22.063618 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 10 08:24:22.063618 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:24:22.063618 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 08:24:22.063618 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:24:22.063618 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:22.063618 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 10 08:24:22.066759 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 10 08:24:22.066759 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 10 08:24:22.066759 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 10 08:24:22.066759 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 10 08:24:22.066759 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:24:22.066759 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 08:24:22.066759 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:24:22.066759 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:22.066759 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:24:22.066759 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:24:22.066759 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:24:22.066759 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:24:22.066759 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:24:22.066759 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:24:22.066759 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:24:22.066759 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:24:22.066759 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:24:22.067996 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:24:22.067996 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 10 08:24:22.067996 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 10 08:24:22.067996 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:24:22.067996 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:24:22.067996 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 10 08:24:22.067996 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 10 08:24:22.067996 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 43070d3d2c65450f9daa7cbf11207731. Aug 10 08:24:22.067996 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:24:22.068637 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6648e839-e29e-47ea-aed6-f40cfdddf12d None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:22.068637 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6648e839-e29e-47ea-aed6-f40cfdddf12d None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:22.068637 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6648e839-e29e-47ea-aed6-f40cfdddf12d None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:22.068637 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 468/940] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:24:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:24:22.068968 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 473/941] 10.222.0.57 () {60 vars in 1140 bytes} [Wed Aug 10 08:24:21 2022] POST /identity/v3/projects => generated 262 bytes in 241 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 10 08:24:22.069073 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6648e839-e29e-47ea-aed6-f40cfdddf12d None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:22.069168 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a4ed3e-8e1f-40b0-9039-302467425e6f None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:22.081805 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:22.167768 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a4ed3e-8e1f-40b0-9039-302467425e6f None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:22.171576 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6648e839-e29e-47ea-aed6-f40cfdddf12d None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:22.208868 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4467e1de-5183-490d-beed-f467414c70bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:22.209829 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4467e1de-5183-490d-beed-f467414c70bb None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:22.210289 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4467e1de-5183-490d-beed-f467414c70bb None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:22.210737 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4467e1de-5183-490d-beed-f467414c70bb None admin] PATH_INFO: `/v3/services` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:22.210853 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7fc103-f212-4957-b6f2-ea956f0c7e4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:22.211428 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7fc103-f212-4957-b6f2-ea956f0c7e4b None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:22.211709 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7fc103-f212-4957-b6f2-ea956f0c7e4b None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:22.211981 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7fc103-f212-4957-b6f2-ea956f0c7e4b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:22.212215 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4467e1de-5183-490d-beed-f467414c70bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:22.214109 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4467e1de-5183-490d-beed-f467414c70bb None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:22.218710 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7fc103-f212-4957-b6f2-ea956f0c7e4b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:22.220126 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7fc103-f212-4957-b6f2-ea956f0c7e4b None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:22.233578 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 474/942] 10.222.0.57 () {56 vars in 1087 bytes} [Wed Aug 10 08:24:22 2022] GET /identity/v3/services => generated 1497 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:24:22.233578 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5104a08-13b2-4bd9-bcfa-abae9d24a4bf None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:22.233578 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5104a08-13b2-4bd9-bcfa-abae9d24a4bf None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:22.233936 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-7d7fc103-f212-4957-b6f2-ea956f0c7e4b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 08:24:22.233936 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:22.233936 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:24:22.233936 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:24:22.233936 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:24:22.233936 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:24:22.233936 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:24:22.233936 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:24:22.233936 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:24:22.233936 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:24:22.233936 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:24:22.233936 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:24:22.233936 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 08:24:22.233936 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 08:24:22.233936 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 08:24:22.234779 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 08:24:22.234779 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:24:22.234779 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:24:22.234779 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:24:22.234779 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:24:22.234779 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:24:22.234779 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:24:22.234779 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:24:22.234779 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:24:22.234779 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:24:22.234779 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:24:22.234779 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:24:22.234779 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:24:22.234779 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:24:22.234779 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:24:22.234779 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:24:22.234779 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:24:22.235723 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 08:24:22.235723 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 08:24:22.235723 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 08:24:22.235723 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 08:24:22.235723 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 08:24:22.235723 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 08:24:22.235723 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 08:24:22.235723 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:24:22.236203 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5104a08-13b2-4bd9-bcfa-abae9d24a4bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:22.241046 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 469/943] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Aug 10 08:24:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:24:22.241046 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7fc103-f212-4957-b6f2-ea956f0c7e4b None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:22.256171 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-e5104a08-13b2-4bd9-bcfa-abae9d24a4bf None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:24:22.258540 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-e5104a08-13b2-4bd9-bcfa-abae9d24a4bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:24:22.291909 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:22.328736 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:22.343656 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 475/944] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:24:22 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 08:24:22.355813 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7fc103-f212-4957-b6f2-ea956f0c7e4b None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:22.367498 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5104a08-13b2-4bd9-bcfa-abae9d24a4bf None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:22.396637 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb85b015-bce8-4de8-879e-a915b2327aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:22.397293 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb85b015-bce8-4de8-879e-a915b2327aae None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:22.397573 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb85b015-bce8-4de8-879e-a915b2327aae None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:22.397844 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb85b015-bce8-4de8-879e-a915b2327aae None admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:22.398523 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb85b015-bce8-4de8-879e-a915b2327aae None admin] RBAC: Adding query filter params (name=service) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:22.398825 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb85b015-bce8-4de8-879e-a915b2327aae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:22.400287 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb85b015-bce8-4de8-879e-a915b2327aae None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:22.423653 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 470/945] 10.222.0.57 () {56 vars in 1112 bytes} [Wed Aug 10 08:24:22 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:22.496012 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5104a08-13b2-4bd9-bcfa-abae9d24a4bf None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:22.537567 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d788945-ef28-402d-b651-fe05354f9e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:22.538502 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d788945-ef28-402d-b651-fe05354f9e7f None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:22.538964 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d788945-ef28-402d-b651-fe05354f9e7f None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:22.539428 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d788945-ef28-402d-b651-fe05354f9e7f None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:22.540166 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d788945-ef28-402d-b651-fe05354f9e7f None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:22.541841 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d788945-ef28-402d-b651-fe05354f9e7f None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:22.552002 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-0d788945-ef28-402d-b651-fe05354f9e7f None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 08:24:22.552002 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:22.552002 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 08:24:22.552002 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 08:24:22.552002 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 08:24:22.552002 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 08:24:22.552002 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 08:24:22.552002 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 08:24:22.552002 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 08:24:22.552002 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:24:22.552002 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 08:24:22.552002 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:24:22.552002 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:22.552002 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:24:22.552002 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:24:22.552002 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:24:22.553254 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:24:22.553254 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:24:22.553254 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:24:22.553254 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:24:22.553254 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:24:22.553254 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:24:22.553254 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:24:22.553254 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 08:24:22.553254 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 08:24:22.553254 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 08:24:22.553254 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 08:24:22.553254 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:24:22.553254 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:24:22.553254 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:24:22.553254 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:24:22.553254 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:24:22.553254 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:24:22.554401 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:24:22.554401 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:24:22.554401 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:24:22.554401 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:24:22.554401 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:24:22.554401 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:24:22.554401 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:24:22.554401 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:24:22.554401 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:24:22.554401 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:24:22.554401 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 08:24:22.554401 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 08:24:22.554401 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 08:24:22.554401 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:24:22.557055 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 476/946] 10.222.0.57 () {56 vars in 1132 bytes} [Wed Aug 10 08:24:22 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 08:24:22.566008 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb85b015-bce8-4de8-879e-a915b2327aae None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:22.675778 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb85b015-bce8-4de8-879e-a915b2327aae None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:22.712716 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5e92ec-09aa-4f81-8420-02f6eca4b1b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:22.713520 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5e92ec-09aa-4f81-8420-02f6eca4b1b2 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:22.713803 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5e92ec-09aa-4f81-8420-02f6eca4b1b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:22.714122 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5e92ec-09aa-4f81-8420-02f6eca4b1b2 None admin] PATH_INFO: `/v3/services` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:22.714762 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5e92ec-09aa-4f81-8420-02f6eca4b1b2 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:22.715063 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5e92ec-09aa-4f81-8420-02f6eca4b1b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:22.716486 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5e92ec-09aa-4f81-8420-02f6eca4b1b2 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:22.718491 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:22.727288 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 471/947] 10.222.0.57 () {56 vars in 1126 bytes} [Wed Aug 10 08:24:22 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:22.728669 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d788945-ef28-402d-b651-fe05354f9e7f None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:22.757644 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:22.837215 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d788945-ef28-402d-b651-fe05354f9e7f None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:22.873222 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc537936-f0ff-4347-92bf-36bd23de2d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:22.873840 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc537936-f0ff-4347-92bf-36bd23de2d39 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:22.874124 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc537936-f0ff-4347-92bf-36bd23de2d39 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:22.874399 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc537936-f0ff-4347-92bf-36bd23de2d39 None admin] PATH_INFO: `/v3/services` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:22.875031 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc537936-f0ff-4347-92bf-36bd23de2d39 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:22.875340 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc537936-f0ff-4347-92bf-36bd23de2d39 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:22.877352 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc537936-f0ff-4347-92bf-36bd23de2d39 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:22.887193 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 477/948] 10.222.0.57 () {56 vars in 1126 bytes} [Wed Aug 10 08:24:22 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:22.888509 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5e92ec-09aa-4f81-8420-02f6eca4b1b2 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:22.994965 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5e92ec-09aa-4f81-8420-02f6eca4b1b2 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:23.032725 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e04a5a7-0b09-4c0d-9235-88a09dcfdee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:23.033618 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e04a5a7-0b09-4c0d-9235-88a09dcfdee6 None admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:23.033900 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e04a5a7-0b09-4c0d-9235-88a09dcfdee6 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:23.034225 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e04a5a7-0b09-4c0d-9235-88a09dcfdee6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:23.034900 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e04a5a7-0b09-4c0d-9235-88a09dcfdee6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:23.036247 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e04a5a7-0b09-4c0d-9235-88a09dcfdee6 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:23.128310 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 472/949] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 08:24:22 2022] POST /identity/v3/endpoints => generated 347 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:24:23.777894 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba01a1e-c4e4-40fd-8b4c-6e6328ac1c58 None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:23.777894 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba01a1e-c4e4-40fd-8b4c-6e6328ac1c58 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:23.777894 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba01a1e-c4e4-40fd-8b4c-6e6328ac1c58 None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:23.778859 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 478/950] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:24:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:24:23.784512 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43d7476-fa7a-4b3a-8d63-5cbe046049f0 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:23.785281 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43d7476-fa7a-4b3a-8d63-5cbe046049f0 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:23.785705 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43d7476-fa7a-4b3a-8d63-5cbe046049f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:23.804451 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-e43d7476-fa7a-4b3a-8d63-5cbe046049f0 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:24:23.805914 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-e43d7476-fa7a-4b3a-8d63-5cbe046049f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:24:23.857573 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:23.880641 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 473/951] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:24:23 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 08:24:23.904805 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba01a1e-c4e4-40fd-8b4c-6e6328ac1c58 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:23.995008 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:24.010845 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba01a1e-c4e4-40fd-8b4c-6e6328ac1c58 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:24.047033 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063c92e4-c03f-4e94-9d15-e6630c9260f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:24.047587 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063c92e4-c03f-4e94-9d15-e6630c9260f1 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:24.048267 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063c92e4-c03f-4e94-9d15-e6630c9260f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:24.048568 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063c92e4-c03f-4e94-9d15-e6630c9260f1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:24.059323 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063c92e4-c03f-4e94-9d15-e6630c9260f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 08:24:24.063967 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-063c92e4-c03f-4e94-9d15-e6630c9260f1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 08:24:24.063967 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:24.063967 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:24:24.063967 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:24:24.063967 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:24:24.063967 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:24:24.063967 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:24:24.063967 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:24:24.063967 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:24:24.063967 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:24:24.063967 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:24:24.063967 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:24:24.063967 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 08:24:24.063967 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 08:24:24.063967 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 08:24:24.063967 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 08:24:24.065231 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 08:24:24.065231 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 08:24:24.065231 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 08:24:24.065231 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 08:24:24.065231 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:24:24.065231 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:24:24.065231 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:24:24.065231 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:24:24.065231 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:24:24.065231 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:24:24.065231 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:24:24.065231 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:24:24.065231 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:24:24.065231 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:24:24.065231 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:24:24.065231 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:24:24.066157 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:24:24.066157 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:24:24.066157 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:24:24.066157 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:24:24.066157 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 08:24:24.066157 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 08:24:24.066157 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 08:24:24.066157 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 08:24:24.066157 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 08:24:24.066157 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 08:24:24.066157 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 08:24:24.066157 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:24:24.066157 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 479/952] 10.222.0.57 () {56 vars in 1120 bytes} [Wed Aug 10 08:24:23 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:24:24.071368 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43d7476-fa7a-4b3a-8d63-5cbe046049f0 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:24.177161 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43d7476-fa7a-4b3a-8d63-5cbe046049f0 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:24.212571 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ea59da-4804-478c-87a6-59ac1ad780a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:24.213256 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ea59da-4804-478c-87a6-59ac1ad780a2 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:24.213532 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ea59da-4804-478c-87a6-59ac1ad780a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:24.213796 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ea59da-4804-478c-87a6-59ac1ad780a2 None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:24.214462 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ea59da-4804-478c-87a6-59ac1ad780a2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:24.214759 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ea59da-4804-478c-87a6-59ac1ad780a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:24.216121 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ea59da-4804-478c-87a6-59ac1ad780a2 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:24.225301 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 474/953] 10.222.0.57 () {56 vars in 1115 bytes} [Wed Aug 10 08:24:24 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:24.230798 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063c92e4-c03f-4e94-9d15-e6630c9260f1 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:24.340781 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063c92e4-c03f-4e94-9d15-e6630c9260f1 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:24.361792 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ebd679-9441-4a79-b5b8-289883b53b55 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:24.362073 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ebd679-9441-4a79-b5b8-289883b53b55 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:24.362348 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ebd679-9441-4a79-b5b8-289883b53b55 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:24.363857 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 475/954] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:24:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:24:24.368657 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35097ef5-f6c4-4472-9f0d-088eab4213e7 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:24.370322 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35097ef5-f6c4-4472-9f0d-088eab4213e7 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:24.370322 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35097ef5-f6c4-4472-9f0d-088eab4213e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:24.377298 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa68be01-da9f-4347-b395-7d478e9520eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:24.377951 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa68be01-da9f-4347-b395-7d478e9520eb None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:24.378238 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa68be01-da9f-4347-b395-7d478e9520eb None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:24.378508 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa68be01-da9f-4347-b395-7d478e9520eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:24.392021 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa68be01-da9f-4347-b395-7d478e9520eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:24.394475 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa68be01-da9f-4347-b395-7d478e9520eb None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:24.400121 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-35097ef5-f6c4-4472-9f0d-088eab4213e7 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:24:24.401400 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-35097ef5-f6c4-4472-9f0d-088eab4213e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:24:24.411727 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:24.424179 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-fa68be01-da9f-4347-b395-7d478e9520eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:24:24.424179 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:24.424179 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:24:24.424179 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:24:24.424179 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:24:24.424179 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:24:24.424179 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:24:24.424179 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:24:24.424179 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:24:24.424179 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:24:24.424179 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:24:24.424179 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:24:24.424179 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 08:24:24.424179 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 08:24:24.424179 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 08:24:24.424179 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 08:24:24.425233 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:24:24.425233 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:24:24.425233 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:24:24.425233 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:24:24.425233 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:24:24.425233 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:24:24.425233 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:24:24.425233 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:24:24.425233 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:24:24.425233 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:24:24.425233 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:24:24.425233 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:24:24.425233 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:24:24.425233 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:24:24.425233 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:24:24.425233 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:24:24.425233 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 08:24:24.426154 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 08:24:24.426154 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:24:24.426154 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:24:24.428873 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 480/955] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:24:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:24:24.434589 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa68be01-da9f-4347-b395-7d478e9520eb None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:24.452097 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:24.481407 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 476/956] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:24:24 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 08:24:24.505259 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35097ef5-f6c4-4472-9f0d-088eab4213e7 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:24.547464 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa68be01-da9f-4347-b395-7d478e9520eb None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:24.584766 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17988fdc-2b12-41e9-82b2-9d58335dbcce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:24.585689 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17988fdc-2b12-41e9-82b2-9d58335dbcce None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:24.586119 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17988fdc-2b12-41e9-82b2-9d58335dbcce None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:24.586550 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17988fdc-2b12-41e9-82b2-9d58335dbcce None admin] PATH_INFO: `/v3/domains` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:24.587352 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17988fdc-2b12-41e9-82b2-9d58335dbcce None admin] RBAC: Adding query filter params (name=Default) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:24.587823 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17988fdc-2b12-41e9-82b2-9d58335dbcce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:24.589376 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17988fdc-2b12-41e9-82b2-9d58335dbcce None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:24.606731 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 481/957] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Aug 10 08:24:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:24.607833 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17988fdc-2b12-41e9-82b2-9d58335dbcce None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:24.610340 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35097ef5-f6c4-4472-9f0d-088eab4213e7 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:24.646253 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba4a176-5751-4dbc-8553-af45b39138e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:24.646890 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba4a176-5751-4dbc-8553-af45b39138e7 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:24.647154 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba4a176-5751-4dbc-8553-af45b39138e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:24.647419 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba4a176-5751-4dbc-8553-af45b39138e7 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:24.647991 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba4a176-5751-4dbc-8553-af45b39138e7 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:24.649394 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba4a176-5751-4dbc-8553-af45b39138e7 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:24.658238 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-3ba4a176-5751-4dbc-8553-af45b39138e7 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 08:24:24.658238 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:24.658238 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 08:24:24.658238 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 08:24:24.658238 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 08:24:24.658238 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 08:24:24.658238 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 08:24:24.658238 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 08:24:24.658238 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 08:24:24.658238 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:24:24.658238 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 08:24:24.658238 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:24:24.658238 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:24.658238 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:24:24.658238 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:24:24.658238 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:24:24.658238 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:24:24.659374 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:24:24.659374 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:24:24.659374 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:24:24.659374 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:24:24.659374 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:24:24.659374 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:24:24.659374 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 08:24:24.659374 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 08:24:24.659374 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 08:24:24.659374 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 08:24:24.659374 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:24:24.659374 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:24:24.659374 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:24:24.659374 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:24:24.659374 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:24:24.659374 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:24:24.660268 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:24:24.660268 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:24:24.660268 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:24:24.660268 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:24:24.660268 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:24:24.660268 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:24:24.660268 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:24:24.660268 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:24:24.660268 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:24:24.660268 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:24:24.660268 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 08:24:24.660268 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 08:24:24.660268 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 08:24:24.660268 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:24:24.664195 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 477/958] 10.222.0.57 () {56 vars in 1126 bytes} [Wed Aug 10 08:24:24 2022] GET /identity/v3/services/object-store => generated 93 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 08:24:24.665412 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d4de70-9336-4800-8679-61dc7f950606 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:24.665695 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d4de70-9336-4800-8679-61dc7f950606 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:24.665966 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d4de70-9336-4800-8679-61dc7f950606 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:24.667329 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 478/959] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:24:24 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:24:24.667881 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d4de70-9336-4800-8679-61dc7f950606 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:24.675981 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:24.713143 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:24.722211 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17988fdc-2b12-41e9-82b2-9d58335dbcce None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:24.762089 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f297fe-9bda-4051-aac4-ae7b7358b294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:24.763005 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f297fe-9bda-4051-aac4-ae7b7358b294 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:24.763461 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f297fe-9bda-4051-aac4-ae7b7358b294 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:24.763880 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f297fe-9bda-4051-aac4-ae7b7358b294 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:24.771528 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f297fe-9bda-4051-aac4-ae7b7358b294 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:24.773178 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f297fe-9bda-4051-aac4-ae7b7358b294 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:24.778453 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d4de70-9336-4800-8679-61dc7f950606 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:24.785015 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-46f297fe-9bda-4051-aac4-ae7b7358b294 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 08:24:24.785015 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:24.785015 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:24:24.785015 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:24:24.785015 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:24:24.785015 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:24:24.785015 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:24:24.785015 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:24:24.785015 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:24:24.785015 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:24:24.785015 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:24:24.785015 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:24:24.785015 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 08:24:24.785015 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 08:24:24.785015 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 08:24:24.785015 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 08:24:24.786336 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:24:24.786336 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:24:24.786336 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 08:24:24.786336 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 08:24:24.786336 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 08:24:24.786336 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 08:24:24.786336 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:24:24.786336 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:24:24.786336 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:24:24.786336 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:24:24.786336 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:24:24.786336 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:24:24.786336 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:24:24.786336 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:24:24.786336 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:24:24.786336 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:24:24.786336 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:24:24.786336 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:24:24.787365 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:24:24.787365 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:24:24.787365 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 08:24:24.787365 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 08:24:24.787365 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 08:24:24.787365 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 08:24:24.787365 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 08:24:24.787365 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 08:24:24.787365 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 08:24:24.787365 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:24:24.787365 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 482/960] 10.222.0.57 () {56 vars in 1117 bytes} [Wed Aug 10 08:24:24 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:24:24.787365 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8393da6c-57f8-48eb-9b5a-cb400f20a883 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:24.788073 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8393da6c-57f8-48eb-9b5a-cb400f20a883 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:24.788073 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8393da6c-57f8-48eb-9b5a-cb400f20a883 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:24.806000 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-8393da6c-57f8-48eb-9b5a-cb400f20a883 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:24:24.806976 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-8393da6c-57f8-48eb-9b5a-cb400f20a883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:24:24.821150 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72af188-6da1-4303-9077-8328eab6e6eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:24.821800 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72af188-6da1-4303-9077-8328eab6e6eb None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:24.822108 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72af188-6da1-4303-9077-8328eab6e6eb None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:24.822444 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72af188-6da1-4303-9077-8328eab6e6eb None admin] PATH_INFO: `/v3/services` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:24.823106 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72af188-6da1-4303-9077-8328eab6e6eb None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:24.823406 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72af188-6da1-4303-9077-8328eab6e6eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:24.824760 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72af188-6da1-4303-9077-8328eab6e6eb None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:24.829564 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 479/961] 10.222.0.57 () {56 vars in 1122 bytes} [Wed Aug 10 08:24:24 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:24.830347 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72af188-6da1-4303-9077-8328eab6e6eb None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:24.886321 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 483/962] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:24:24 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 08:24:24.887403 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8393da6c-57f8-48eb-9b5a-cb400f20a883 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:24.947170 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:24.949323 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72af188-6da1-4303-9077-8328eab6e6eb None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:24.950866 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:24.983926 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94b0b67-9d34-4205-8b40-d3dc07ea8614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:24.984524 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94b0b67-9d34-4205-8b40-d3dc07ea8614 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:24.984805 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94b0b67-9d34-4205-8b40-d3dc07ea8614 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:24.985104 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94b0b67-9d34-4205-8b40-d3dc07ea8614 None admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:24.985849 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94b0b67-9d34-4205-8b40-d3dc07ea8614 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:24.986675 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94b0b67-9d34-4205-8b40-d3dc07ea8614 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:24.988090 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94b0b67-9d34-4205-8b40-d3dc07ea8614 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:25.001803 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8393da6c-57f8-48eb-9b5a-cb400f20a883 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:25.010335 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 480/963] 10.222.0.57 () {56 vars in 1149 bytes} [Wed Aug 10 08:24:24 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:25.010335 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94b0b67-9d34-4205-8b40-d3dc07ea8614 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:25.041978 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c085846c-0f1f-418e-ae35-7d3c523326fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:25.042585 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c085846c-0f1f-418e-ae35-7d3c523326fa None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:25.042860 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c085846c-0f1f-418e-ae35-7d3c523326fa None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:25.043126 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c085846c-0f1f-418e-ae35-7d3c523326fa None admin] PATH_INFO: `/v3/services` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:25.043748 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c085846c-0f1f-418e-ae35-7d3c523326fa None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:25.044047 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c085846c-0f1f-418e-ae35-7d3c523326fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:25.045458 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c085846c-0f1f-418e-ae35-7d3c523326fa None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:25.056335 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 484/964] 10.222.0.57 () {56 vars in 1122 bytes} [Wed Aug 10 08:24:24 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:25.056335 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c085846c-0f1f-418e-ae35-7d3c523326fa None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:25.117712 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94b0b67-9d34-4205-8b40-d3dc07ea8614 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:25.157443 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ef13bb-b3ca-4525-96b2-a905ff23f043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:25.157783 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c085846c-0f1f-418e-ae35-7d3c523326fa None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:25.158015 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ef13bb-b3ca-4525-96b2-a905ff23f043 None admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:25.158295 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ef13bb-b3ca-4525-96b2-a905ff23f043 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:25.158565 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ef13bb-b3ca-4525-96b2-a905ff23f043 None admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:25.159307 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ef13bb-b3ca-4525-96b2-a905ff23f043 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:25.160859 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ef13bb-b3ca-4525-96b2-a905ff23f043 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:25.163432 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: WARNING keystone.server.flask.common [None req-17ef13bb-b3ca-4525-96b2-a905ff23f043 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 10 08:24:25.171606 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:25.192610 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23de9286-d538-4760-b493-0ff6aaade7df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:25.193243 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23de9286-d538-4760-b493-0ff6aaade7df None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:25.193701 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23de9286-d538-4760-b493-0ff6aaade7df None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:25.193831 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23de9286-d538-4760-b493-0ff6aaade7df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:25.211493 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.sql.core [None req-17ef13bb-b3ca-4525-96b2-a905ff23f043 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 10 08:24:25.211493 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 10 08:24:25.211493 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [parameters: {'user_id': 'e0138aba24de4ed394d8140a94a075e2', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 10 08:24:25.211493 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80462) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 10 08:24:25.218030 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-17ef13bb-b3ca-4525-96b2-a905ff23f043 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 10 08:24:25.218030 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:25.218030 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 10 08:24:25.218030 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 10 08:24:25.218030 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 10 08:24:25.218030 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 10 08:24:25.218030 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 10 08:24:25.218030 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application result = self._query(query) Aug 10 08:24:25.218030 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 10 08:24:25.218030 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application conn.query(q) Aug 10 08:24:25.218030 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 10 08:24:25.218030 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 10 08:24:25.218030 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 10 08:24:25.218030 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application result.read() Aug 10 08:24:25.218030 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 10 08:24:25.220690 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 10 08:24:25.220690 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 10 08:24:25.220690 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 10 08:24:25.220690 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 10 08:24:25.220690 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 10 08:24:25.220690 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 10 08:24:25.220690 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 10 08:24:25.220690 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 10 08:24:25.220690 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:24:25.220690 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 10 08:24:25.220690 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:24:25.220690 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:25.220690 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 10 08:24:25.220690 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 10 08:24:25.220690 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 10 08:24:25.220690 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 10 08:24:25.220690 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 10 08:24:25.220690 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application next(self.gen) Aug 10 08:24:25.221733 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 10 08:24:25.221733 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application yield resource Aug 10 08:24:25.221733 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 10 08:24:25.221733 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application next(self.gen) Aug 10 08:24:25.221733 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 10 08:24:25.221733 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application self.session.rollback() Aug 10 08:24:25.221733 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 10 08:24:25.221733 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application self.force_reraise() Aug 10 08:24:25.221733 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 10 08:24:25.221733 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise self.value Aug 10 08:24:25.221733 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 10 08:24:25.221733 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 10 08:24:25.221733 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 10 08:24:25.221733 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application session.commit() Aug 10 08:24:25.221733 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 10 08:24:25.221733 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 10 08:24:25.221733 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 10 08:24:25.222720 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application self._prepare_impl() Aug 10 08:24:25.222720 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 10 08:24:25.222720 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application self.session.flush() Aug 10 08:24:25.222720 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 10 08:24:25.222720 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application self._flush(objects) Aug 10 08:24:25.222720 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 10 08:24:25.222720 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 10 08:24:25.222720 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 10 08:24:25.222720 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application compat.raise_( Aug 10 08:24:25.222720 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 10 08:24:25.222720 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception Aug 10 08:24:25.222720 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 10 08:24:25.222720 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application flush_context.execute() Aug 10 08:24:25.222720 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 10 08:24:25.222720 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rec.execute(self) Aug 10 08:24:25.222720 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 10 08:24:25.222720 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 10 08:24:25.223634 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 10 08:24:25.223634 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 10 08:24:25.223634 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 10 08:24:25.223634 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 10 08:24:25.223634 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 10 08:24:25.223634 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 10 08:24:25.223634 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 10 08:24:25.223634 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 10 08:24:25.223634 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 10 08:24:25.223634 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 10 08:24:25.223634 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 10 08:24:25.223634 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 10 08:24:25.223634 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 10 08:24:25.223634 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 10 08:24:25.223634 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 10 08:24:25.223634 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception Aug 10 08:24:25.225625 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 10 08:24:25.225625 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 10 08:24:25.225625 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 10 08:24:25.225625 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 10 08:24:25.225625 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 10 08:24:25.225625 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application result = self._query(query) Aug 10 08:24:25.225625 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 10 08:24:25.225625 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application conn.query(q) Aug 10 08:24:25.225625 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 10 08:24:25.225625 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 10 08:24:25.225625 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 10 08:24:25.225625 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application result.read() Aug 10 08:24:25.225625 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 10 08:24:25.225625 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 10 08:24:25.225625 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 10 08:24:25.225625 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 10 08:24:25.227373 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 10 08:24:25.227373 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 10 08:24:25.227373 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 10 08:24:25.227373 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 10 08:24:25.227373 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 10 08:24:25.227373 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 10 08:24:25.227373 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [parameters: {'user_id': 'e0138aba24de4ed394d8140a94a075e2', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 10 08:24:25.227373 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 10 08:24:25.227373 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:24:25.227373 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 08:24:25.227373 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:24:25.227373 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:25.227373 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:24:25.227373 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:24:25.227373 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:24:25.230032 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:24:25.230032 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:24:25.230032 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:24:25.230032 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:24:25.230032 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:24:25.230032 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:24:25.230032 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:24:25.230032 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 10 08:24:25.230032 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 10 08:24:25.230032 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:24:25.230032 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:24:25.230032 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 08:24:25.230032 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 08:24:25.230032 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 08:24:25.230032 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 08:24:25.230032 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 10 08:24:25.230032 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 10 08:24:25.232219 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 10 08:24:25.232219 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 10 08:24:25.232219 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 10 08:24:25.232219 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 10 08:24:25.232219 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 10 08:24:25.232219 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:24:25.232219 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 481/965] 10.222.0.57 () {60 vars in 1131 bytes} [Wed Aug 10 08:24:25 2022] POST /identity/v3/users => generated 160 bytes in 213 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:24:25.232219 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ef13bb-b3ca-4525-96b2-a905ff23f043 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:25.248727 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:25.263264 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23de9286-d538-4760-b493-0ff6aaade7df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:25.264724 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23de9286-d538-4760-b493-0ff6aaade7df None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:25.325293 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ef13bb-b3ca-4525-96b2-a905ff23f043 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:25.359006 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa6476d-53c9-42ad-9d68-64e9e57d05ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:25.359608 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa6476d-53c9-42ad-9d68-64e9e57d05ef None admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:25.359887 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa6476d-53c9-42ad-9d68-64e9e57d05ef None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:25.360156 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa6476d-53c9-42ad-9d68-64e9e57d05ef None admin] PATH_INFO: `/v3/endpoints` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:25.361345 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa6476d-53c9-42ad-9d68-64e9e57d05ef None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:25.362972 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 485/966] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Aug 10 08:24:25 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 08:24:25.363127 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa6476d-53c9-42ad-9d68-64e9e57d05ef None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:25.364291 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aa17b9-d185-4dea-a526-3270b37cb176 None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:25.364486 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aa17b9-d185-4dea-a526-3270b37cb176 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:25.364752 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aa17b9-d185-4dea-a526-3270b37cb176 None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:25.369071 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 486/967] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:24:25 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:24:25.369071 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aa17b9-d185-4dea-a526-3270b37cb176 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:25.384828 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:25.467631 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aa17b9-d185-4dea-a526-3270b37cb176 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:25.476927 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 482/968] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 08:24:25 2022] POST /identity/v3/endpoints => generated 349 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:24:25.477838 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa6476d-53c9-42ad-9d68-64e9e57d05ef None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:25.516659 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034acc39-c0bb-44ca-9999-7d0d1907cc33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:25.517552 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034acc39-c0bb-44ca-9999-7d0d1907cc33 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:25.517912 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034acc39-c0bb-44ca-9999-7d0d1907cc33 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:25.518182 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034acc39-c0bb-44ca-9999-7d0d1907cc33 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:25.525933 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034acc39-c0bb-44ca-9999-7d0d1907cc33 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:25.527416 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034acc39-c0bb-44ca-9999-7d0d1907cc33 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:25.540992 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-034acc39-c0bb-44ca-9999-7d0d1907cc33 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 10 08:24:25.540992 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:25.540992 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:24:25.540992 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:24:25.540992 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:24:25.540992 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:24:25.540992 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:24:25.540992 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:24:25.540992 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:24:25.540992 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:24:25.540992 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:24:25.540992 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:24:25.540992 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 08:24:25.540992 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 08:24:25.540992 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 08:24:25.540992 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 08:24:25.542026 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:24:25.542026 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:24:25.542026 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 08:24:25.542026 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 08:24:25.542026 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 08:24:25.542026 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 08:24:25.542026 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:24:25.542026 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:24:25.542026 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:24:25.542026 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:24:25.542026 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:24:25.542026 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:24:25.542026 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:24:25.542026 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:24:25.542026 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:24:25.542026 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:24:25.542026 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:24:25.542026 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:24:25.543008 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:24:25.543008 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:24:25.543008 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 08:24:25.543008 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 08:24:25.543008 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 08:24:25.543008 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 08:24:25.543008 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 08:24:25.543008 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 08:24:25.543008 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 10 08:24:25.543008 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:24:25.543008 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 487/969] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Aug 10 08:24:25 2022] GET /identity/v3/users/cinder => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:24:25.543617 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948f9494-2797-4149-a194-624a68230695 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:25.543617 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948f9494-2797-4149-a194-624a68230695 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:25.544833 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948f9494-2797-4149-a194-624a68230695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:25.565662 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-948f9494-2797-4149-a194-624a68230695 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:24:25.566944 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-948f9494-2797-4149-a194-624a68230695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:24:25.595910 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:25.623436 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa6476d-53c9-42ad-9d68-64e9e57d05ef None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:25.639036 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:25.645897 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 488/970] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:24:25 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 08:24:25.646708 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948f9494-2797-4149-a194-624a68230695 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:25.663094 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d12b12-36ad-4f75-87c8-072cbc365202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:25.663821 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d12b12-36ad-4f75-87c8-072cbc365202 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:25.664107 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d12b12-36ad-4f75-87c8-072cbc365202 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:25.664375 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d12b12-36ad-4f75-87c8-072cbc365202 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:25.679679 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d12b12-36ad-4f75-87c8-072cbc365202 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:25.681156 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d12b12-36ad-4f75-87c8-072cbc365202 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:25.697407 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-72d12b12-36ad-4f75-87c8-072cbc365202 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:24:25.697407 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:25.697407 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:24:25.697407 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:24:25.697407 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:24:25.697407 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:24:25.697407 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:24:25.697407 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:24:25.697407 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:24:25.697407 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:24:25.697407 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:24:25.697407 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:24:25.697407 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 08:24:25.697407 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 08:24:25.697407 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 08:24:25.697407 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 08:24:25.698411 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:24:25.698411 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:24:25.698411 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:24:25.698411 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:24:25.698411 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:24:25.698411 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:24:25.698411 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:24:25.698411 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:24:25.698411 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:24:25.698411 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:24:25.698411 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:24:25.698411 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:24:25.698411 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:24:25.698411 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:24:25.698411 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:24:25.698411 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:24:25.698411 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 08:24:25.699320 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 08:24:25.699320 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:24:25.699320 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:24:25.699320 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 483/971] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:24:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:24:25.699320 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d12b12-36ad-4f75-87c8-072cbc365202 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:25.760182 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948f9494-2797-4149-a194-624a68230695 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:25.795868 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b5bc06-bc27-4496-96ed-45f63c9fd728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:25.796552 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b5bc06-bc27-4496-96ed-45f63c9fd728 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:25.796790 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b5bc06-bc27-4496-96ed-45f63c9fd728 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:25.797036 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b5bc06-bc27-4496-96ed-45f63c9fd728 None admin] PATH_INFO: `/v3/users` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:25.798044 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b5bc06-bc27-4496-96ed-45f63c9fd728 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:25.798044 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b5bc06-bc27-4496-96ed-45f63c9fd728 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:25.800014 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b5bc06-bc27-4496-96ed-45f63c9fd728 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:25.810670 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d12b12-36ad-4f75-87c8-072cbc365202 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:25.844954 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 489/972] 10.222.0.57 () {56 vars in 1101 bytes} [Wed Aug 10 08:24:25 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:25.844954 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b5bc06-bc27-4496-96ed-45f63c9fd728 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:25.850736 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:25.857315 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2de30b-d2d9-407b-858f-42f775774c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:25.858047 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2de30b-d2d9-407b-858f-42f775774c83 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:25.858521 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2de30b-d2d9-407b-858f-42f775774c83 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:25.858854 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2de30b-d2d9-407b-858f-42f775774c83 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:25.859527 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2de30b-d2d9-407b-858f-42f775774c83 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:25.861202 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2de30b-d2d9-407b-858f-42f775774c83 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:25.875324 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-ee2de30b-d2d9-407b-858f-42f775774c83 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 08:24:25.875324 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:25.875324 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 08:24:25.875324 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 08:24:25.875324 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 08:24:25.875324 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 08:24:25.875324 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 08:24:25.875324 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 08:24:25.875324 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 08:24:25.875324 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:24:25.875324 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 08:24:25.875324 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:24:25.875324 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:25.875324 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:24:25.875324 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:24:25.875324 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:24:25.875324 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:24:25.876997 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:24:25.876997 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:24:25.876997 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:24:25.876997 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:24:25.876997 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:24:25.876997 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:24:25.876997 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 08:24:25.876997 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 08:24:25.876997 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 08:24:25.876997 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 08:24:25.876997 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:24:25.876997 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:24:25.876997 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:24:25.876997 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:24:25.876997 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:24:25.876997 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:24:25.878338 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:24:25.878338 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:24:25.878338 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:24:25.878338 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:24:25.878338 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:24:25.878338 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:24:25.878338 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:24:25.878338 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:24:25.878338 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:24:25.878338 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:24:25.878338 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 08:24:25.878338 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 08:24:25.878338 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 08:24:25.878338 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:24:25.881106 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 484/973] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Aug 10 08:24:25 2022] GET /identity/v3/services/compute => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:24:25.883656 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2de30b-d2d9-407b-858f-42f775774c83 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:25.915801 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:25.977726 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b5bc06-bc27-4496-96ed-45f63c9fd728 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:26.020179 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2de30b-d2d9-407b-858f-42f775774c83 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:26.024062 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09210e1d-06c7-4c89-ab32-7933a808cacd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:26.024932 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09210e1d-06c7-4c89-ab32-7933a808cacd None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:26.025419 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09210e1d-06c7-4c89-ab32-7933a808cacd None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:26.025867 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09210e1d-06c7-4c89-ab32-7933a808cacd None admin] PATH_INFO: `/v3/domains` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:26.026699 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09210e1d-06c7-4c89-ab32-7933a808cacd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:26.027181 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09210e1d-06c7-4c89-ab32-7933a808cacd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:26.028724 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09210e1d-06c7-4c89-ab32-7933a808cacd None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:26.045578 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 490/974] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Aug 10 08:24:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:26.054014 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09210e1d-06c7-4c89-ab32-7933a808cacd None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:26.064667 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f098c7a-7f89-4bc7-aa74-d2aa0e14e675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:26.065554 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f098c7a-7f89-4bc7-aa74-d2aa0e14e675 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:26.065982 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f098c7a-7f89-4bc7-aa74-d2aa0e14e675 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:26.066422 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f098c7a-7f89-4bc7-aa74-d2aa0e14e675 None admin] PATH_INFO: `/v3/services` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:26.067200 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f098c7a-7f89-4bc7-aa74-d2aa0e14e675 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:26.067669 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f098c7a-7f89-4bc7-aa74-d2aa0e14e675 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:26.069806 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f098c7a-7f89-4bc7-aa74-d2aa0e14e675 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:26.075430 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 485/975] 10.222.0.57 () {56 vars in 1112 bytes} [Wed Aug 10 08:24:25 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:24:26.082003 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f098c7a-7f89-4bc7-aa74-d2aa0e14e675 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:26.161508 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:26.189426 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09210e1d-06c7-4c89-ab32-7933a808cacd None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:26.205105 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f098c7a-7f89-4bc7-aa74-d2aa0e14e675 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:26.212622 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:26.235106 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498aecbf-9f6f-49d6-865d-8a7395bb6a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:26.236123 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498aecbf-9f6f-49d6-865d-8a7395bb6a22 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:26.236423 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498aecbf-9f6f-49d6-865d-8a7395bb6a22 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:26.236642 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498aecbf-9f6f-49d6-865d-8a7395bb6a22 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:26.241520 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271fa5d3-97cf-4e65-a5d6-4df7069b1d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:26.242127 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271fa5d3-97cf-4e65-a5d6-4df7069b1d40 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:26.242405 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271fa5d3-97cf-4e65-a5d6-4df7069b1d40 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:26.242832 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271fa5d3-97cf-4e65-a5d6-4df7069b1d40 None admin] PATH_INFO: `/v3/services` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:26.243559 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498aecbf-9f6f-49d6-865d-8a7395bb6a22 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:26.243667 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271fa5d3-97cf-4e65-a5d6-4df7069b1d40 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:26.244002 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271fa5d3-97cf-4e65-a5d6-4df7069b1d40 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:26.245278 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498aecbf-9f6f-49d6-865d-8a7395bb6a22 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:26.245566 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271fa5d3-97cf-4e65-a5d6-4df7069b1d40 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:26.250516 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 486/976] 10.222.0.57 () {56 vars in 1112 bytes} [Wed Aug 10 08:24:26 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:24:26.261241 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-498aecbf-9f6f-49d6-865d-8a7395bb6a22 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 08:24:26.261241 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:26.261241 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:24:26.261241 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:24:26.261241 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:24:26.261241 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:24:26.261241 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:24:26.261241 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:24:26.261241 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:24:26.261241 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:24:26.261241 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:24:26.261241 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:24:26.261241 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 08:24:26.261241 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 08:24:26.261241 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 08:24:26.262096 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 08:24:26.262096 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:24:26.262096 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:24:26.262096 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:24:26.262096 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:24:26.262096 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:24:26.262096 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:24:26.262096 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:24:26.262096 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:24:26.262096 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:24:26.262096 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:24:26.262096 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:24:26.262096 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:24:26.262096 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:24:26.262096 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:24:26.262096 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:24:26.262096 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:24:26.263019 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 08:24:26.263019 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 08:24:26.263019 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 08:24:26.263019 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 08:24:26.263019 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 08:24:26.263019 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 08:24:26.263019 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 08:24:26.263019 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:24:26.263019 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 491/977] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Aug 10 08:24:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 08:24:26.266669 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271fa5d3-97cf-4e65-a5d6-4df7069b1d40 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:26.378719 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271fa5d3-97cf-4e65-a5d6-4df7069b1d40 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:26.414389 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:26.426244 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f628748-923b-4389-ad4f-cffb656da311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:26.426863 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f628748-923b-4389-ad4f-cffb656da311 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:26.427823 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f628748-923b-4389-ad4f-cffb656da311 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:26.427823 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f628748-923b-4389-ad4f-cffb656da311 None admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:26.428103 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f628748-923b-4389-ad4f-cffb656da311 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:26.428409 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f628748-923b-4389-ad4f-cffb656da311 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:26.429909 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f628748-923b-4389-ad4f-cffb656da311 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:26.442529 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 487/978] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Aug 10 08:24:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:26.448583 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498aecbf-9f6f-49d6-865d-8a7395bb6a22 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:26.551441 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498aecbf-9f6f-49d6-865d-8a7395bb6a22 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:26.586233 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb58ca3-e03d-4722-84a9-8f261da82e03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:26.586838 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb58ca3-e03d-4722-84a9-8f261da82e03 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:26.587118 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb58ca3-e03d-4722-84a9-8f261da82e03 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:26.587387 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb58ca3-e03d-4722-84a9-8f261da82e03 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:26.588107 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb58ca3-e03d-4722-84a9-8f261da82e03 None admin] RBAC: Adding query filter params (role.id=c30407dbef394307b4d8d3dc9ca4bcaa, scope.project.id=b49e381419e940278bcb6111899589b5, user.id=c140d31bd2b84c48a170e3ca90a041ce) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:26.588410 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb58ca3-e03d-4722-84a9-8f261da82e03 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:26.589860 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb58ca3-e03d-4722-84a9-8f261da82e03 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:26.595642 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 492/979] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Aug 10 08:24:26 2022] GET /identity/v3/role_assignments?user.id=c140d31bd2b84c48a170e3ca90a041ce&scope.project.id=b49e381419e940278bcb6111899589b5&role.id=c30407dbef394307b4d8d3dc9ca4bcaa => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:27.602022 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460e6439-c884-4dfb-9c31-c88daed59a2b None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:27.602542 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460e6439-c884-4dfb-9c31-c88daed59a2b None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:27.602542 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460e6439-c884-4dfb-9c31-c88daed59a2b None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:27.604288 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 488/980] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:24:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 08:24:27.609789 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d538dca-47fc-4981-9f2c-958f39b69d36 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:27.610187 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d538dca-47fc-4981-9f2c-958f39b69d36 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:27.610354 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d538dca-47fc-4981-9f2c-958f39b69d36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:27.629690 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-0d538dca-47fc-4981-9f2c-958f39b69d36 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:24:27.630860 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-0d538dca-47fc-4981-9f2c-958f39b69d36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:24:27.667972 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:27.701366 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 493/981] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:24:27 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 08:24:27.726374 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460e6439-c884-4dfb-9c31-c88daed59a2b None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:27.830320 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460e6439-c884-4dfb-9c31-c88daed59a2b None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:27.865412 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e667910-0b94-4dad-8dfb-0475e59a627d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:27.866404 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e667910-0b94-4dad-8dfb-0475e59a627d None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:27.866892 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e667910-0b94-4dad-8dfb-0475e59a627d None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:27.867365 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e667910-0b94-4dad-8dfb-0475e59a627d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:27.868156 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e667910-0b94-4dad-8dfb-0475e59a627d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:27.870022 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e667910-0b94-4dad-8dfb-0475e59a627d None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:27.879210 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-1e667910-0b94-4dad-8dfb-0475e59a627d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 08:24:27.879210 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:27.879210 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 08:24:27.879210 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 08:24:27.879210 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 08:24:27.879210 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 08:24:27.879210 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 08:24:27.879210 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 08:24:27.879210 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 08:24:27.879210 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:24:27.879210 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 08:24:27.879210 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:24:27.879210 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:27.879210 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:24:27.879210 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:24:27.879210 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:24:27.879210 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:24:27.880220 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:24:27.880220 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:24:27.880220 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:24:27.880220 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:24:27.880220 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:24:27.880220 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:24:27.880220 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 08:24:27.880220 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 08:24:27.880220 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 08:24:27.880220 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 08:24:27.880220 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:24:27.880220 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:24:27.880220 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:24:27.880220 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:24:27.880220 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:24:27.880220 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:24:27.881223 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:24:27.881223 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:24:27.881223 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:24:27.881223 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:24:27.881223 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:24:27.881223 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:24:27.881223 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:24:27.881223 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:24:27.881223 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:24:27.881223 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:24:27.881223 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 08:24:27.881223 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 08:24:27.881223 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 08:24:27.881223 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:24:27.886932 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 489/982] 10.222.0.57 () {56 vars in 1126 bytes} [Wed Aug 10 08:24:27 2022] GET /identity/v3/services/object-store => generated 93 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 08:24:27.887926 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d538dca-47fc-4981-9f2c-958f39b69d36 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:27.988494 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d538dca-47fc-4981-9f2c-958f39b69d36 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:28.023237 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c4763c-a811-45a5-bff7-e0bf0d51366b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:28.023840 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c4763c-a811-45a5-bff7-e0bf0d51366b None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:28.024115 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c4763c-a811-45a5-bff7-e0bf0d51366b None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:28.024382 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c4763c-a811-45a5-bff7-e0bf0d51366b None admin] PATH_INFO: `/v3/services` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:28.025023 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c4763c-a811-45a5-bff7-e0bf0d51366b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:28.025324 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c4763c-a811-45a5-bff7-e0bf0d51366b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:28.026683 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c4763c-a811-45a5-bff7-e0bf0d51366b None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:28.035988 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 494/983] 10.222.0.57 () {56 vars in 1122 bytes} [Wed Aug 10 08:24:27 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:28.037351 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e667910-0b94-4dad-8dfb-0475e59a627d None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:28.088687 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:28.140978 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e667910-0b94-4dad-8dfb-0475e59a627d None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:28.175926 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f317793-ad1e-48d9-88ed-4b53eb5ebb73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:28.176532 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f317793-ad1e-48d9-88ed-4b53eb5ebb73 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:28.176811 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f317793-ad1e-48d9-88ed-4b53eb5ebb73 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:28.177109 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f317793-ad1e-48d9-88ed-4b53eb5ebb73 None admin] PATH_INFO: `/v3/services` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:28.177846 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f317793-ad1e-48d9-88ed-4b53eb5ebb73 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:28.178150 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f317793-ad1e-48d9-88ed-4b53eb5ebb73 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:28.179511 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f317793-ad1e-48d9-88ed-4b53eb5ebb73 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:28.189741 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 490/984] 10.222.0.57 () {56 vars in 1122 bytes} [Wed Aug 10 08:24:28 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:28.191234 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c4763c-a811-45a5-bff7-e0bf0d51366b None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:28.227487 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:28.297957 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c4763c-a811-45a5-bff7-e0bf0d51366b None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:28.315914 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28865ba-05b4-4155-84cd-ca6a82dab2e5 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:28.316209 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28865ba-05b4-4155-84cd-ca6a82dab2e5 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:28.316462 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28865ba-05b4-4155-84cd-ca6a82dab2e5 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:28.317955 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 491/985] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:24:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 08:24:28.322592 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc4230a-3b82-4a9e-a422-823525bcea1d None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:28.322871 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc4230a-3b82-4a9e-a422-823525bcea1d None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:28.323141 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc4230a-3b82-4a9e-a422-823525bcea1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:28.337414 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab1391d-6bfc-4eff-8db0-886eed1c092a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:28.338025 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab1391d-6bfc-4eff-8db0-886eed1c092a None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:28.338303 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab1391d-6bfc-4eff-8db0-886eed1c092a None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:28.338573 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab1391d-6bfc-4eff-8db0-886eed1c092a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:28.339281 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab1391d-6bfc-4eff-8db0-886eed1c092a None admin] RBAC: Adding query filter params (interface=admin, service_id=db11dbab82b34ba69856d999b49c60c7, region_id=RegionOne) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:28.339586 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab1391d-6bfc-4eff-8db0-886eed1c092a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:28.340973 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab1391d-6bfc-4eff-8db0-886eed1c092a None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:28.342029 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-4bc4230a-3b82-4a9e-a422-823525bcea1d None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:24:28.343597 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-4bc4230a-3b82-4a9e-a422-823525bcea1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:24:28.350756 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 495/986] 10.222.0.57 () {56 vars in 1249 bytes} [Wed Aug 10 08:24:28 2022] GET /identity/v3/endpoints?service_id=db11dbab82b34ba69856d999b49c60c7&interface=admin®ion_id=RegionOne => generated 193 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:24:28.351767 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab1391d-6bfc-4eff-8db0-886eed1c092a None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:28.433703 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 492/987] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:24:28 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 08:24:28.452205 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc4230a-3b82-4a9e-a422-823525bcea1d None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:28.460476 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:28.464643 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab1391d-6bfc-4eff-8db0-886eed1c092a None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:28.497125 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:28.499448 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c4b926-ef92-40b4-8742-7a969e815f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:28.500236 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c4b926-ef92-40b4-8742-7a969e815f77 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:28.500517 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c4b926-ef92-40b4-8742-7a969e815f77 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:28.500787 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c4b926-ef92-40b4-8742-7a969e815f77 None admin] PATH_INFO: `/v3/services` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:28.502049 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c4b926-ef92-40b4-8742-7a969e815f77 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:28.503524 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c4b926-ef92-40b4-8742-7a969e815f77 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:28.512164 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 496/988] 10.222.0.57 () {56 vars in 1087 bytes} [Wed Aug 10 08:24:28 2022] GET /identity/v3/services => generated 1497 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:24:28.573394 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc4230a-3b82-4a9e-a422-823525bcea1d None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:28.613238 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f0d02b-9477-4e95-bd26-62320d02baf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:28.614649 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f0d02b-9477-4e95-bd26-62320d02baf9 None admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:28.615123 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f0d02b-9477-4e95-bd26-62320d02baf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:28.615574 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f0d02b-9477-4e95-bd26-62320d02baf9 None admin] PATH_INFO: `/v3/services` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:28.616415 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f0d02b-9477-4e95-bd26-62320d02baf9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:28.618122 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f0d02b-9477-4e95-bd26-62320d02baf9 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:28.660883 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d2f59b-a6f2-4377-b5c8-63824d57c5f3 None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:28.661453 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d2f59b-a6f2-4377-b5c8-63824d57c5f3 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:28.661549 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d2f59b-a6f2-4377-b5c8-63824d57c5f3 None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:28.662852 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 497/989] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:24:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 08:24:28.667913 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92adde7e-0e82-4d9f-876e-9567c31207db None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:28.668218 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92adde7e-0e82-4d9f-876e-9567c31207db None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:28.668489 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92adde7e-0e82-4d9f-876e-9567c31207db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:28.686821 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 493/990] 10.222.0.57 () {60 vars in 1141 bytes} [Wed Aug 10 08:24:28 2022] POST /identity/v3/services => generated 242 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 08:24:28.687219 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-92adde7e-0e82-4d9f-876e-9567c31207db None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:24:28.688405 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-92adde7e-0e82-4d9f-876e-9567c31207db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:24:28.773759 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 498/991] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:24:28 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 08:24:28.797707 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f0d02b-9477-4e95-bd26-62320d02baf9 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:28.906558 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:28.909935 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f0d02b-9477-4e95-bd26-62320d02baf9 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:28.944089 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a483e7-3eaf-4515-a521-4575a663ef16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:28.944726 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a483e7-3eaf-4515-a521-4575a663ef16 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:28.945030 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a483e7-3eaf-4515-a521-4575a663ef16 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:28.947098 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a483e7-3eaf-4515-a521-4575a663ef16 None admin] PATH_INFO: `/v3/services/image` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:28.947098 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a483e7-3eaf-4515-a521-4575a663ef16 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:28.947281 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a483e7-3eaf-4515-a521-4575a663ef16 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:28.956699 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-60a483e7-3eaf-4515-a521-4575a663ef16 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 08:24:28.956699 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:28.956699 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 08:24:28.956699 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 08:24:28.956699 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 08:24:28.956699 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 08:24:28.956699 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 08:24:28.956699 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 08:24:28.956699 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 08:24:28.956699 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:24:28.956699 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 08:24:28.956699 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:24:28.956699 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:28.956699 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:24:28.956699 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:24:28.956699 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:24:28.956699 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:24:28.957930 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:24:28.957930 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:24:28.957930 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:24:28.957930 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:24:28.957930 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:24:28.957930 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:24:28.957930 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 08:24:28.957930 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 08:24:28.957930 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 08:24:28.957930 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 08:24:28.957930 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:24:28.957930 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:24:28.957930 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:24:28.957930 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:24:28.957930 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:24:28.957930 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:24:28.959041 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:24:28.959041 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:24:28.959041 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:24:28.959041 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:24:28.959041 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:24:28.959041 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:24:28.959041 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:24:28.959041 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:24:28.959041 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:24:28.959041 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:24:28.959041 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 08:24:28.959041 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 08:24:28.959041 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 08:24:28.959041 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:24:28.959041 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 494/992] 10.222.0.57 () {56 vars in 1105 bytes} [Wed Aug 10 08:24:28 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:24:28.963837 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92adde7e-0e82-4d9f-876e-9567c31207db None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:29.055458 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:29.068392 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92adde7e-0e82-4d9f-876e-9567c31207db None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:29.102526 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd1b7b6-237b-437f-8dfb-7702e932d08b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:29.103200 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd1b7b6-237b-437f-8dfb-7702e932d08b None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:29.103487 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd1b7b6-237b-437f-8dfb-7702e932d08b None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:29.103759 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd1b7b6-237b-437f-8dfb-7702e932d08b None admin] PATH_INFO: `/v3/services` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:29.104384 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd1b7b6-237b-437f-8dfb-7702e932d08b None admin] RBAC: Adding query filter params (name=image) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:29.104687 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd1b7b6-237b-437f-8dfb-7702e932d08b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:29.106086 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd1b7b6-237b-437f-8dfb-7702e932d08b None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:29.115798 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 499/993] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:24:28 2022] GET /identity/v3/services?name=image => generated 122 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:24:29.117068 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a483e7-3eaf-4515-a521-4575a663ef16 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:29.222025 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a483e7-3eaf-4515-a521-4575a663ef16 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:29.259239 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3bc303-580c-47a3-ae9a-8a508dbd0598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:29.259849 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3bc303-580c-47a3-ae9a-8a508dbd0598 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:29.260119 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3bc303-580c-47a3-ae9a-8a508dbd0598 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:29.260392 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3bc303-580c-47a3-ae9a-8a508dbd0598 None admin] PATH_INFO: `/v3/services` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:29.261048 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3bc303-580c-47a3-ae9a-8a508dbd0598 None admin] RBAC: Adding query filter params (type=image) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:29.261352 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3bc303-580c-47a3-ae9a-8a508dbd0598 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:29.262713 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3bc303-580c-47a3-ae9a-8a508dbd0598 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:29.273364 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 495/994] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:24:29 2022] GET /identity/v3/services?type=image => generated 122 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:30.549081 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5496ad51-be62-4a7b-88af-ef31cb653aa0 None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:30.549732 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5496ad51-be62-4a7b-88af-ef31cb653aa0 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:30.549732 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5496ad51-be62-4a7b-88af-ef31cb653aa0 None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:30.550995 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 500/995] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:24:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:24:30.556081 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9723258-ed52-41a7-83e1-fe1f968e901d None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:30.556480 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9723258-ed52-41a7-83e1-fe1f968e901d None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:30.556620 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9723258-ed52-41a7-83e1-fe1f968e901d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:30.575252 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-e9723258-ed52-41a7-83e1-fe1f968e901d None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:24:30.576421 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-e9723258-ed52-41a7-83e1-fe1f968e901d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:24:30.600617 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:30.646097 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 496/996] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:24:30 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 08:24:30.668245 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5496ad51-be62-4a7b-88af-ef31cb653aa0 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:30.767572 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0bb3b7-cc5e-49d4-aab3-d2de3ebc8ea4 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:30.767850 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0bb3b7-cc5e-49d4-aab3-d2de3ebc8ea4 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:30.768125 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0bb3b7-cc5e-49d4-aab3-d2de3ebc8ea4 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:30.769511 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 497/997] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:24:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 08:24:30.774517 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f2e021-5737-4f80-b910-af8a770b1b0d None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:30.774802 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f2e021-5737-4f80-b910-af8a770b1b0d None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:30.775077 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f2e021-5737-4f80-b910-af8a770b1b0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:30.778543 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5496ad51-be62-4a7b-88af-ef31cb653aa0 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:30.793854 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-65f2e021-5737-4f80-b910-af8a770b1b0d None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:24:30.795056 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-65f2e021-5737-4f80-b910-af8a770b1b0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:24:30.816310 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8ea2ba-7a8b-466b-b1b7-03b0bc8da61b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:30.816973 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8ea2ba-7a8b-466b-b1b7-03b0bc8da61b None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:30.817263 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8ea2ba-7a8b-466b-b1b7-03b0bc8da61b None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:30.817537 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8ea2ba-7a8b-466b-b1b7-03b0bc8da61b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:30.818121 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8ea2ba-7a8b-466b-b1b7-03b0bc8da61b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:30.819522 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8ea2ba-7a8b-466b-b1b7-03b0bc8da61b None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:30.828103 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-6e8ea2ba-7a8b-466b-b1b7-03b0bc8da61b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 08:24:30.828103 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:30.828103 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 08:24:30.828103 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 08:24:30.828103 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 08:24:30.828103 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 08:24:30.828103 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 08:24:30.828103 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 08:24:30.828103 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 08:24:30.828103 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:24:30.828103 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 08:24:30.828103 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:24:30.828103 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:30.828103 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:24:30.828103 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:24:30.828103 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:24:30.828103 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:24:30.829136 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:24:30.829136 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:24:30.829136 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:24:30.829136 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:24:30.829136 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:24:30.829136 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:24:30.829136 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 08:24:30.829136 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 08:24:30.829136 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 08:24:30.829136 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 08:24:30.829136 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:24:30.829136 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:24:30.829136 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:24:30.829136 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:24:30.829136 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:24:30.829136 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:24:30.830796 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:24:30.830796 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:24:30.830796 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:24:30.830796 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:24:30.830796 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:24:30.830796 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:24:30.830796 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:24:30.830796 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:24:30.830796 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:24:30.830796 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:24:30.830796 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 08:24:30.830796 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 08:24:30.830796 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 08:24:30.830796 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:24:30.830796 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 501/998] 10.222.0.57 () {56 vars in 1126 bytes} [Wed Aug 10 08:24:30 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:24:30.835043 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8ea2ba-7a8b-466b-b1b7-03b0bc8da61b None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:30.853644 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:30.877024 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 498/999] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:24:30 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 08:24:30.899509 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f2e021-5737-4f80-b910-af8a770b1b0d None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:30.940116 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8ea2ba-7a8b-466b-b1b7-03b0bc8da61b None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:30.976481 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb287c1-1603-4758-8182-509a641dce6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:30.977119 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb287c1-1603-4758-8182-509a641dce6e None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:30.977477 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb287c1-1603-4758-8182-509a641dce6e None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:30.977668 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb287c1-1603-4758-8182-509a641dce6e None admin] PATH_INFO: `/v3/services` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:30.978289 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb287c1-1603-4758-8182-509a641dce6e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:30.979301 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb287c1-1603-4758-8182-509a641dce6e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:30.979956 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb287c1-1603-4758-8182-509a641dce6e None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:30.984967 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 502/1000] 10.222.0.57 () {56 vars in 1122 bytes} [Wed Aug 10 08:24:30 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:24:30.991229 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb287c1-1603-4758-8182-509a641dce6e None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:30.998971 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:31.012022 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f2e021-5737-4f80-b910-af8a770b1b0d None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:31.047942 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511f76b3-fa87-4b44-b8c3-d1ef715e0fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:31.048638 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511f76b3-fa87-4b44-b8c3-d1ef715e0fc0 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:31.048904 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511f76b3-fa87-4b44-b8c3-d1ef715e0fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:31.049217 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511f76b3-fa87-4b44-b8c3-d1ef715e0fc0 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:31.049804 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511f76b3-fa87-4b44-b8c3-d1ef715e0fc0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:31.051245 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511f76b3-fa87-4b44-b8c3-d1ef715e0fc0 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:31.060812 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-511f76b3-fa87-4b44-b8c3-d1ef715e0fc0 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 08:24:31.060812 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:31.060812 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 08:24:31.060812 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 08:24:31.060812 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 08:24:31.060812 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 08:24:31.060812 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 08:24:31.060812 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 08:24:31.060812 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 08:24:31.060812 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:24:31.060812 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 08:24:31.060812 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:24:31.060812 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:31.060812 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:24:31.060812 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:24:31.060812 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:24:31.060812 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:24:31.062074 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:24:31.062074 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:24:31.062074 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:24:31.062074 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:24:31.062074 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:24:31.062074 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:24:31.062074 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 08:24:31.062074 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 08:24:31.062074 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 08:24:31.062074 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 08:24:31.062074 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:24:31.062074 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:24:31.062074 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:24:31.062074 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:24:31.062074 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:24:31.062074 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:24:31.062948 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:24:31.062948 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:24:31.062948 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:24:31.062948 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:24:31.062948 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:24:31.062948 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:24:31.062948 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:24:31.062948 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:24:31.062948 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:24:31.062948 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:24:31.062948 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 08:24:31.062948 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 08:24:31.062948 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 08:24:31.062948 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:24:31.062948 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 499/1001] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Aug 10 08:24:30 2022] GET /identity/v3/services/compute => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:24:31.069789 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511f76b3-fa87-4b44-b8c3-d1ef715e0fc0 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:31.085171 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:31.106255 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb287c1-1603-4758-8182-509a641dce6e None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:31.143326 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ecfb58-3369-470d-a812-c3b70441f222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:31.143929 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ecfb58-3369-470d-a812-c3b70441f222 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:31.144202 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ecfb58-3369-470d-a812-c3b70441f222 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:31.144468 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ecfb58-3369-470d-a812-c3b70441f222 None admin] PATH_INFO: `/v3/services` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:31.145122 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ecfb58-3369-470d-a812-c3b70441f222 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:31.145420 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ecfb58-3369-470d-a812-c3b70441f222 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:31.146901 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ecfb58-3369-470d-a812-c3b70441f222 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:31.156823 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 503/1002] 10.222.0.57 () {56 vars in 1122 bytes} [Wed Aug 10 08:24:30 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:24:31.157900 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ecfb58-3369-470d-a812-c3b70441f222 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:31.175625 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511f76b3-fa87-4b44-b8c3-d1ef715e0fc0 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:31.213994 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80d5331-852d-4e5c-b921-2d2ad383340f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:31.214930 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80d5331-852d-4e5c-b921-2d2ad383340f None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:31.215568 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80d5331-852d-4e5c-b921-2d2ad383340f None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:31.216179 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80d5331-852d-4e5c-b921-2d2ad383340f None admin] PATH_INFO: `/v3/services` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:31.216998 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80d5331-852d-4e5c-b921-2d2ad383340f None admin] RBAC: Adding query filter params (name=compute) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:31.217404 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80d5331-852d-4e5c-b921-2d2ad383340f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:31.218889 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80d5331-852d-4e5c-b921-2d2ad383340f None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:31.229246 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 500/1003] 10.222.0.57 () {56 vars in 1112 bytes} [Wed Aug 10 08:24:31 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:24:31.231284 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80d5331-852d-4e5c-b921-2d2ad383340f None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:31.252017 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:31.266808 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ecfb58-3369-470d-a812-c3b70441f222 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:31.303188 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9a37e2-5705-4b8e-aadc-cc86d6a5bd9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:31.303815 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9a37e2-5705-4b8e-aadc-cc86d6a5bd9b None admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:31.308280 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9a37e2-5705-4b8e-aadc-cc86d6a5bd9b None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:31.308280 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9a37e2-5705-4b8e-aadc-cc86d6a5bd9b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:31.308280 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9a37e2-5705-4b8e-aadc-cc86d6a5bd9b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:31.308280 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9a37e2-5705-4b8e-aadc-cc86d6a5bd9b None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:31.337333 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80d5331-852d-4e5c-b921-2d2ad383340f None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:31.370568 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8728f8c2-6c76-4c8c-88c4-ba70eb9ffc6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:31.371171 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8728f8c2-6c76-4c8c-88c4-ba70eb9ffc6d None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:31.371452 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8728f8c2-6c76-4c8c-88c4-ba70eb9ffc6d None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:31.371721 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8728f8c2-6c76-4c8c-88c4-ba70eb9ffc6d None admin] PATH_INFO: `/v3/services` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:31.372344 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8728f8c2-6c76-4c8c-88c4-ba70eb9ffc6d None admin] RBAC: Adding query filter params (type=compute) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:31.372645 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8728f8c2-6c76-4c8c-88c4-ba70eb9ffc6d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:31.374039 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8728f8c2-6c76-4c8c-88c4-ba70eb9ffc6d None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:31.383763 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 504/1004] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 08:24:31 2022] POST /identity/v3/endpoints => generated 325 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 08:24:31.383763 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880c3101-ecb7-4026-a5ce-43539655190f None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:31.383763 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880c3101-ecb7-4026-a5ce-43539655190f None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:31.383763 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880c3101-ecb7-4026-a5ce-43539655190f None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:31.384198 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 501/1005] 10.222.0.57 () {56 vars in 1112 bytes} [Wed Aug 10 08:24:31 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:31.384278 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 505/1006] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:24:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:24:31.385694 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8728f8c2-6c76-4c8c-88c4-ba70eb9ffc6d None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:31.394487 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0288fec-a674-49f9-85d5-a2d0073af40e None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:31.394771 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0288fec-a674-49f9-85d5-a2d0073af40e None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:31.403310 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0288fec-a674-49f9-85d5-a2d0073af40e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:31.406427 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:31.419005 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-f0288fec-a674-49f9-85d5-a2d0073af40e None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:24:31.420217 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-f0288fec-a674-49f9-85d5-a2d0073af40e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:24:31.498923 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 506/1007] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:24:31 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 08:24:31.523088 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0288fec-a674-49f9-85d5-a2d0073af40e None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:31.619556 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:31.641020 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8728f8c2-6c76-4c8c-88c4-ba70eb9ffc6d None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:31.646795 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0288fec-a674-49f9-85d5-a2d0073af40e None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:31.679624 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0add23f-dfda-4bb5-a436-a0e1ffa1cf60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:31.680233 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0add23f-dfda-4bb5-a436-a0e1ffa1cf60 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:31.680514 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0add23f-dfda-4bb5-a436-a0e1ffa1cf60 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:31.680621 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:31.680813 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0add23f-dfda-4bb5-a436-a0e1ffa1cf60 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:31.681521 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0add23f-dfda-4bb5-a436-a0e1ffa1cf60 None admin] RBAC: Adding query filter params (interface=public, service_id=a2fd55f72c644b2e9a0ade25716dec73, region_id=RegionOne) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:31.681828 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0add23f-dfda-4bb5-a436-a0e1ffa1cf60 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:31.683229 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0add23f-dfda-4bb5-a436-a0e1ffa1cf60 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:31.686456 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4c4b30-21ab-4ced-86b5-ee16cae13d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:31.687127 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c4b30-21ab-4ced-86b5-ee16cae13d65 None admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:31.687334 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c4b30-21ab-4ced-86b5-ee16cae13d65 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:31.687649 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c4b30-21ab-4ced-86b5-ee16cae13d65 None admin] PATH_INFO: `/v3/services` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:31.688255 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4c4b30-21ab-4ced-86b5-ee16cae13d65 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:31.689272 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 502/1008] 10.222.0.57 () {56 vars in 1251 bytes} [Wed Aug 10 08:24:31 2022] GET /identity/v3/endpoints?service_id=a2fd55f72c644b2e9a0ade25716dec73&interface=public®ion_id=RegionOne => generated 194 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:31.690193 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4c4b30-21ab-4ced-86b5-ee16cae13d65 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:31.694840 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0add23f-dfda-4bb5-a436-a0e1ffa1cf60 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:31.773033 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 507/1009] 10.222.0.57 () {60 vars in 1141 bytes} [Wed Aug 10 08:24:31 2022] POST /identity/v3/services => generated 242 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 08:24:31.804854 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0add23f-dfda-4bb5-a436-a0e1ffa1cf60 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:31.849322 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766d4dee-9e31-48a7-b661-615a6006ffa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:31.849933 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766d4dee-9e31-48a7-b661-615a6006ffa8 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:31.850211 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766d4dee-9e31-48a7-b661-615a6006ffa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:31.850478 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766d4dee-9e31-48a7-b661-615a6006ffa8 None admin] PATH_INFO: `/v3/services` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:31.851085 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766d4dee-9e31-48a7-b661-615a6006ffa8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:31.852472 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766d4dee-9e31-48a7-b661-615a6006ffa8 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:31.859839 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 503/1010] 10.222.0.57 () {56 vars in 1087 bytes} [Wed Aug 10 08:24:31 2022] GET /identity/v3/services => generated 1957 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 08:24:33.441858 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a32b0c-82ce-48a4-a0dc-935c6be048e4 None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:33.442399 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a32b0c-82ce-48a4-a0dc-935c6be048e4 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:33.442399 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a32b0c-82ce-48a4-a0dc-935c6be048e4 None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:33.443785 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 508/1011] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:24:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:24:33.448859 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc937a96-7d11-4d4b-9dc1-d52139cb215b None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:33.449295 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc937a96-7d11-4d4b-9dc1-d52139cb215b None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:33.449460 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc937a96-7d11-4d4b-9dc1-d52139cb215b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:33.457039 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:33.467822 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-dc937a96-7d11-4d4b-9dc1-d52139cb215b None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:24:33.469030 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-dc937a96-7d11-4d4b-9dc1-d52139cb215b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:24:33.538999 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 504/1012] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:24:33 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 08:24:33.562295 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a32b0c-82ce-48a4-a0dc-935c6be048e4 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:33.665519 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a32b0c-82ce-48a4-a0dc-935c6be048e4 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:33.699743 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049e1afb-be3d-4440-8d8f-9b89e34bee49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:33.700371 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049e1afb-be3d-4440-8d8f-9b89e34bee49 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:33.700648 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049e1afb-be3d-4440-8d8f-9b89e34bee49 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:33.700944 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049e1afb-be3d-4440-8d8f-9b89e34bee49 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:33.702049 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049e1afb-be3d-4440-8d8f-9b89e34bee49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:33.703476 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049e1afb-be3d-4440-8d8f-9b89e34bee49 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:33.709984 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 509/1013] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:24:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:33.711124 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc937a96-7d11-4d4b-9dc1-d52139cb215b None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:33.812452 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc937a96-7d11-4d4b-9dc1-d52139cb215b None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:33.818517 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d475bc-af9c-47e5-ab76-734ab59526b5 None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:33.818801 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d475bc-af9c-47e5-ab76-734ab59526b5 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:33.819076 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d475bc-af9c-47e5-ab76-734ab59526b5 None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:33.820534 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 510/1014] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:24:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 08:24:33.825139 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aceb3c-fe66-4b3e-9251-fdd03980408e None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:33.825426 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aceb3c-fe66-4b3e-9251-fdd03980408e None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:33.825724 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aceb3c-fe66-4b3e-9251-fdd03980408e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:33.844161 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-c8aceb3c-fe66-4b3e-9251-fdd03980408e None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:24:33.845964 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-c8aceb3c-fe66-4b3e-9251-fdd03980408e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:24:33.849343 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5149c834-3b1a-415e-8896-28f799e03e51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:33.849964 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5149c834-3b1a-415e-8896-28f799e03e51 None admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:33.850242 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5149c834-3b1a-415e-8896-28f799e03e51 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:33.850512 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5149c834-3b1a-415e-8896-28f799e03e51 None admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:33.851155 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5149c834-3b1a-415e-8896-28f799e03e51 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:33.852581 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5149c834-3b1a-415e-8896-28f799e03e51 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:33.857777 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:33.902391 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:33.916166 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 511/1015] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:24:33 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 08:24:33.917506 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6820f334-ebab-456e-bcab-09df63c5da38 None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:33.917786 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6820f334-ebab-456e-bcab-09df63c5da38 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:33.918055 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6820f334-ebab-456e-bcab-09df63c5da38 None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:33.919417 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 512/1016] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:24:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:24:33.924873 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786cdc76-6415-4776-b693-3652ecba492d None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:33.925184 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786cdc76-6415-4776-b693-3652ecba492d None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:33.925460 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786cdc76-6415-4776-b693-3652ecba492d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:33.943533 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-786cdc76-6415-4776-b693-3652ecba492d None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:24:33.944703 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-786cdc76-6415-4776-b693-3652ecba492d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:24:34.000283 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 505/1017] 10.222.0.57 () {60 vars in 1141 bytes} [Wed Aug 10 08:24:33 2022] POST /identity/v3/projects => generated 311 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 08:24:34.001393 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5149c834-3b1a-415e-8896-28f799e03e51 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:34.015776 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 513/1018] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:24:33 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 08:24:34.039741 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786cdc76-6415-4776-b693-3652ecba492d None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:34.119691 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5149c834-3b1a-415e-8896-28f799e03e51 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:34.159415 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc602cf-249e-4773-8960-8e1eec990e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:34.160209 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc602cf-249e-4773-8960-8e1eec990e66 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:34.160493 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc602cf-249e-4773-8960-8e1eec990e66 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:34.160767 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc602cf-249e-4773-8960-8e1eec990e66 None admin] PATH_INFO: `/v3/services/image` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:34.161415 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc602cf-249e-4773-8960-8e1eec990e66 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:34.162787 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc602cf-249e-4773-8960-8e1eec990e66 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:34.164298 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786cdc76-6415-4776-b693-3652ecba492d None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:34.173821 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-9bc602cf-249e-4773-8960-8e1eec990e66 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 08:24:34.173821 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:34.173821 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 08:24:34.173821 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 08:24:34.173821 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 08:24:34.173821 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 08:24:34.173821 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 08:24:34.173821 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 08:24:34.173821 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 08:24:34.173821 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:24:34.173821 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 08:24:34.173821 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:24:34.173821 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:34.173821 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:24:34.173821 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:24:34.173821 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:24:34.173821 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:24:34.174909 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:24:34.174909 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:24:34.174909 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:24:34.174909 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:24:34.174909 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:24:34.174909 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:24:34.174909 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 08:24:34.174909 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 08:24:34.174909 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 08:24:34.174909 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 08:24:34.174909 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:24:34.174909 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:24:34.174909 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:24:34.174909 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:24:34.174909 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:24:34.174909 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:24:34.176589 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:24:34.176589 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:24:34.176589 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:24:34.176589 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:24:34.176589 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:24:34.176589 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:24:34.176589 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:24:34.176589 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:24:34.176589 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:24:34.176589 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:24:34.176589 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 08:24:34.176589 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 08:24:34.176589 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 08:24:34.176589 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:24:34.176589 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 506/1019] 10.222.0.57 () {56 vars in 1105 bytes} [Wed Aug 10 08:24:34 2022] GET /identity/v3/services/image => generated 86 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:24:34.180963 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc602cf-249e-4773-8960-8e1eec990e66 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:34.186308 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:34.205851 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1798999-1f44-4b70-a06a-58a6d07df69e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:34.206576 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1798999-1f44-4b70-a06a-58a6d07df69e None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:34.206758 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1798999-1f44-4b70-a06a-58a6d07df69e None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:34.207025 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1798999-1f44-4b70-a06a-58a6d07df69e None admin] PATH_INFO: `/v3/services/compute` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:34.207802 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1798999-1f44-4b70-a06a-58a6d07df69e None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:34.209054 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1798999-1f44-4b70-a06a-58a6d07df69e None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:34.218668 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-a1798999-1f44-4b70-a06a-58a6d07df69e None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 08:24:34.218668 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:34.218668 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 08:24:34.218668 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 08:24:34.218668 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 08:24:34.218668 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 08:24:34.218668 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 08:24:34.218668 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 08:24:34.218668 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 08:24:34.218668 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:24:34.218668 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 08:24:34.218668 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:24:34.218668 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:34.218668 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:24:34.218668 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:24:34.218668 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:24:34.218668 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:24:34.219692 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:24:34.219692 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:24:34.219692 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:24:34.219692 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:24:34.219692 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:24:34.219692 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:24:34.219692 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 08:24:34.219692 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 08:24:34.219692 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 08:24:34.219692 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 08:24:34.219692 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:24:34.219692 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:24:34.219692 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:24:34.219692 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:24:34.219692 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:24:34.219692 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:24:34.220641 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:24:34.220641 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:24:34.220641 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:24:34.220641 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:24:34.220641 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:24:34.220641 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:24:34.220641 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:24:34.220641 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:24:34.220641 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:24:34.220641 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:24:34.220641 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 08:24:34.220641 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 08:24:34.220641 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 08:24:34.220641 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:24:34.220641 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 514/1020] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Aug 10 08:24:34 2022] GET /identity/v3/services/compute => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 08:24:34.227112 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1798999-1f44-4b70-a06a-58a6d07df69e None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:34.244803 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:34.301434 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc602cf-249e-4773-8960-8e1eec990e66 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:34.337790 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26216aec-9010-49e7-93b9-a460efb647ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:34.338436 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26216aec-9010-49e7-93b9-a460efb647ed None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:34.338721 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26216aec-9010-49e7-93b9-a460efb647ed None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:34.339007 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26216aec-9010-49e7-93b9-a460efb647ed None admin] PATH_INFO: `/v3/services` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:34.339628 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26216aec-9010-49e7-93b9-a460efb647ed None admin] RBAC: Adding query filter params (name=image) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:34.339932 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26216aec-9010-49e7-93b9-a460efb647ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:34.341320 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26216aec-9010-49e7-93b9-a460efb647ed None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:34.345195 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1798999-1f44-4b70-a06a-58a6d07df69e None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:34.346489 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 507/1021] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:24:34 2022] GET /identity/v3/services?name=image => generated 122 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:24:34.351798 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26216aec-9010-49e7-93b9-a460efb647ed None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:34.395338 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54bbd75-92bd-495d-9886-3a4dcc334686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:34.395963 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54bbd75-92bd-495d-9886-3a4dcc334686 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:34.396246 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54bbd75-92bd-495d-9886-3a4dcc334686 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:34.396844 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54bbd75-92bd-495d-9886-3a4dcc334686 None admin] PATH_INFO: `/v3/services` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:34.398334 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54bbd75-92bd-495d-9886-3a4dcc334686 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:34.398334 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54bbd75-92bd-495d-9886-3a4dcc334686 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:34.400871 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54bbd75-92bd-495d-9886-3a4dcc334686 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:34.411353 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 515/1022] 10.222.0.57 () {56 vars in 1112 bytes} [Wed Aug 10 08:24:34 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:34.413435 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54bbd75-92bd-495d-9886-3a4dcc334686 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:34.457480 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26216aec-9010-49e7-93b9-a460efb647ed None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:34.465438 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:34.473416 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:34.493911 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ae2f7f-19d4-476b-a453-1078b3eddb8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:34.494787 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ae2f7f-19d4-476b-a453-1078b3eddb8d None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:34.495071 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ae2f7f-19d4-476b-a453-1078b3eddb8d None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:34.495345 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ae2f7f-19d4-476b-a453-1078b3eddb8d None admin] PATH_INFO: `/v3/services` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:34.496029 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ae2f7f-19d4-476b-a453-1078b3eddb8d None admin] RBAC: Adding query filter params (type=image) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:34.496333 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ae2f7f-19d4-476b-a453-1078b3eddb8d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:34.497751 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ae2f7f-19d4-476b-a453-1078b3eddb8d None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:34.502821 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 508/1023] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:24:34 2022] GET /identity/v3/services?type=image => generated 350 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:24:34.508786 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ae2f7f-19d4-476b-a453-1078b3eddb8d None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:34.531050 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54bbd75-92bd-495d-9886-3a4dcc334686 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:34.569542 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e362c203-bcef-4a6f-9a70-d270adb725c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:34.570149 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e362c203-bcef-4a6f-9a70-d270adb725c9 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:34.570424 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e362c203-bcef-4a6f-9a70-d270adb725c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:34.570690 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e362c203-bcef-4a6f-9a70-d270adb725c9 None admin] PATH_INFO: `/v3/services` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:34.571309 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e362c203-bcef-4a6f-9a70-d270adb725c9 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:34.571771 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e362c203-bcef-4a6f-9a70-d270adb725c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:34.573439 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e362c203-bcef-4a6f-9a70-d270adb725c9 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:34.578954 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 516/1024] 10.222.0.57 () {56 vars in 1112 bytes} [Wed Aug 10 08:24:34 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:24:34.584790 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e362c203-bcef-4a6f-9a70-d270adb725c9 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:34.618731 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ae2f7f-19d4-476b-a453-1078b3eddb8d None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:34.654631 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d062524e-7de6-43fd-b11a-8789be07430f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:34.655236 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d062524e-7de6-43fd-b11a-8789be07430f None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:34.655517 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d062524e-7de6-43fd-b11a-8789be07430f None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:34.655789 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d062524e-7de6-43fd-b11a-8789be07430f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:34.656487 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d062524e-7de6-43fd-b11a-8789be07430f None admin] RBAC: Adding query filter params (interface=public, service_id=528a6092b2434c3bb82db72180b66373, region_id=RegionOne) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:34.656796 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d062524e-7de6-43fd-b11a-8789be07430f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:34.658914 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d062524e-7de6-43fd-b11a-8789be07430f None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:34.664954 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 509/1025] 10.222.0.57 () {56 vars in 1251 bytes} [Wed Aug 10 08:24:34 2022] GET /identity/v3/endpoints?service_id=528a6092b2434c3bb82db72180b66373&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:24:34.669326 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d062524e-7de6-43fd-b11a-8789be07430f None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:34.698926 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e362c203-bcef-4a6f-9a70-d270adb725c9 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:34.710231 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:34.715103 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:34.733623 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2ed270-f591-4e1b-b3bc-ee92ebb03fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:34.734228 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2ed270-f591-4e1b-b3bc-ee92ebb03fac None admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:34.734505 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2ed270-f591-4e1b-b3bc-ee92ebb03fac None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:34.734774 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2ed270-f591-4e1b-b3bc-ee92ebb03fac None admin] PATH_INFO: `/v3/endpoints` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:34.736060 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2ed270-f591-4e1b-b3bc-ee92ebb03fac None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:34.737578 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2ed270-f591-4e1b-b3bc-ee92ebb03fac None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:34.771765 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d062524e-7de6-43fd-b11a-8789be07430f None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:34.806498 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f08b1e-bf71-4304-8efc-81a1673ee4bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:34.807108 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f08b1e-bf71-4304-8efc-81a1673ee4bf None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:34.807392 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f08b1e-bf71-4304-8efc-81a1673ee4bf None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:34.807664 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f08b1e-bf71-4304-8efc-81a1673ee4bf None admin] PATH_INFO: `/v3/services` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:34.808269 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f08b1e-bf71-4304-8efc-81a1673ee4bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:34.809670 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f08b1e-bf71-4304-8efc-81a1673ee4bf None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:34.816583 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 510/1026] 10.222.0.57 () {56 vars in 1087 bytes} [Wed Aug 10 08:24:34 2022] GET /identity/v3/services => generated 1957 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 08:24:34.831295 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 517/1027] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 08:24:34 2022] POST /identity/v3/endpoints => generated 334 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:24:36.081621 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5351c6b4-9973-4910-8ca9-7e36f6438e47 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:36.082150 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5351c6b4-9973-4910-8ca9-7e36f6438e47 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:36.082231 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5351c6b4-9973-4910-8ca9-7e36f6438e47 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:36.083699 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 511/1028] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:24:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:24:36.088617 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0921072-16e3-4808-b0af-50f8c401753e None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:36.089029 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0921072-16e3-4808-b0af-50f8c401753e None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:36.089227 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0921072-16e3-4808-b0af-50f8c401753e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:36.107484 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-e0921072-16e3-4808-b0af-50f8c401753e None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:24:36.108661 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-e0921072-16e3-4808-b0af-50f8c401753e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:24:36.182839 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 518/1029] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:24:36 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 08:24:36.206864 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5351c6b4-9973-4910-8ca9-7e36f6438e47 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:36.309307 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5351c6b4-9973-4910-8ca9-7e36f6438e47 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:36.316169 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:36.342389 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7ef93c-fbf9-4e1f-a9ee-e2ef0970adb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:36.342950 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7ef93c-fbf9-4e1f-a9ee-e2ef0970adb3 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:36.343228 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7ef93c-fbf9-4e1f-a9ee-e2ef0970adb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:36.343512 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7ef93c-fbf9-4e1f-a9ee-e2ef0970adb3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:36.345280 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7ef93c-fbf9-4e1f-a9ee-e2ef0970adb3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:36.346732 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7ef93c-fbf9-4e1f-a9ee-e2ef0970adb3 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:36.348473 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 512/1030] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:24:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:36.354185 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0921072-16e3-4808-b0af-50f8c401753e None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:36.454977 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:36.463516 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0921072-16e3-4808-b0af-50f8c401753e None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:36.497704 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04d39b4-716f-4511-9b20-03593c00536c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:36.498308 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04d39b4-716f-4511-9b20-03593c00536c None admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:36.498580 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04d39b4-716f-4511-9b20-03593c00536c None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:36.498843 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04d39b4-716f-4511-9b20-03593c00536c None admin] PATH_INFO: `/v3/users` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:36.499510 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04d39b4-716f-4511-9b20-03593c00536c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:36.501534 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04d39b4-716f-4511-9b20-03593c00536c None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:36.584944 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 519/1031] 10.222.0.57 () {60 vars in 1132 bytes} [Wed Aug 10 08:24:36 2022] POST /identity/v3/users => generated 285 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:24:36.864697 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d9cd0d-a094-4376-81ca-52ca11b687dc None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:36.865096 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d9cd0d-a094-4376-81ca-52ca11b687dc None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:36.865299 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d9cd0d-a094-4376-81ca-52ca11b687dc None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:36.866667 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 513/1032] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:24:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:24:36.871992 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566c3523-c2cf-44ba-9766-89164d881252 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:36.872282 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566c3523-c2cf-44ba-9766-89164d881252 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:36.872559 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566c3523-c2cf-44ba-9766-89164d881252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:36.891136 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-566c3523-c2cf-44ba-9766-89164d881252 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:24:36.892624 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-566c3523-c2cf-44ba-9766-89164d881252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:24:36.970684 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e868992-8c9b-43b5-96d4-1bf4038db8b2 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:36.971058 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e868992-8c9b-43b5-96d4-1bf4038db8b2 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:36.971334 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e868992-8c9b-43b5-96d4-1bf4038db8b2 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:36.972844 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 514/1033] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:24:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:24:36.976955 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 520/1034] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:24:36 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 08:24:36.977599 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb248d6-d102-4dce-ac05-628f319e4557 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:36.977884 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb248d6-d102-4dce-ac05-628f319e4557 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:36.978159 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb248d6-d102-4dce-ac05-628f319e4557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:36.996761 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-2bb248d6-d102-4dce-ac05-628f319e4557 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:24:36.997972 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-2bb248d6-d102-4dce-ac05-628f319e4557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:24:36.999661 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566c3523-c2cf-44ba-9766-89164d881252 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:37.067854 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 515/1035] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:24:36 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 08:24:37.082610 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:37.090832 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb248d6-d102-4dce-ac05-628f319e4557 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:37.106870 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566c3523-c2cf-44ba-9766-89164d881252 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:37.146425 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6580976-bcb2-4372-90de-e7600122c185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:37.146984 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6580976-bcb2-4372-90de-e7600122c185 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:37.147259 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6580976-bcb2-4372-90de-e7600122c185 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:37.147529 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6580976-bcb2-4372-90de-e7600122c185 None admin] PATH_INFO: `/v3/services/image` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:37.148113 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6580976-bcb2-4372-90de-e7600122c185 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:37.149528 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6580976-bcb2-4372-90de-e7600122c185 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:37.158353 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-e6580976-bcb2-4372-90de-e7600122c185 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 08:24:37.158353 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:37.158353 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 08:24:37.158353 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 08:24:37.158353 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 08:24:37.158353 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 08:24:37.158353 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 08:24:37.158353 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 08:24:37.158353 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 08:24:37.158353 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:24:37.158353 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 08:24:37.158353 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:24:37.158353 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:37.158353 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:24:37.158353 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:24:37.158353 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:24:37.158353 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:24:37.159555 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:24:37.159555 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:24:37.159555 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:24:37.159555 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:24:37.159555 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:24:37.159555 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:24:37.159555 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 08:24:37.159555 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 08:24:37.159555 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 08:24:37.159555 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 08:24:37.159555 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:24:37.159555 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:24:37.159555 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:24:37.159555 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:24:37.159555 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:24:37.159555 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:24:37.160420 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:24:37.160420 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:24:37.160420 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:24:37.160420 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:24:37.160420 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:24:37.160420 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:24:37.160420 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:24:37.160420 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:24:37.160420 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:24:37.160420 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:24:37.160420 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 08:24:37.160420 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 08:24:37.160420 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 08:24:37.160420 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:24:37.160420 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 521/1036] 10.222.0.57 () {56 vars in 1105 bytes} [Wed Aug 10 08:24:36 2022] GET /identity/v3/services/image => generated 86 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 08:24:37.166100 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6580976-bcb2-4372-90de-e7600122c185 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:37.205145 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb248d6-d102-4dce-ac05-628f319e4557 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:37.207484 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:37.241225 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50baa377-e3f4-4d3e-8678-fb66b03ea2ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:37.241853 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50baa377-e3f4-4d3e-8678-fb66b03ea2ad None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:37.242247 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50baa377-e3f4-4d3e-8678-fb66b03ea2ad None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:37.242531 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50baa377-e3f4-4d3e-8678-fb66b03ea2ad None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:37.255949 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50baa377-e3f4-4d3e-8678-fb66b03ea2ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 08:24:37.260611 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-50baa377-e3f4-4d3e-8678-fb66b03ea2ad None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 08:24:37.260611 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:37.260611 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:24:37.260611 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:24:37.260611 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:24:37.260611 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:24:37.260611 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:24:37.260611 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:24:37.260611 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:24:37.260611 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:24:37.260611 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:24:37.260611 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:24:37.260611 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 08:24:37.260611 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 08:24:37.260611 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 08:24:37.260611 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 08:24:37.261625 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 08:24:37.261625 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 08:24:37.261625 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 08:24:37.261625 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 08:24:37.261625 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:24:37.261625 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:24:37.261625 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:24:37.261625 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:24:37.261625 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:24:37.261625 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:24:37.261625 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:24:37.261625 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:24:37.261625 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:24:37.261625 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:24:37.261625 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:24:37.261625 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:24:37.263056 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:24:37.263056 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:24:37.263056 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:24:37.263056 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:24:37.263056 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 08:24:37.263056 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 08:24:37.263056 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 08:24:37.263056 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 08:24:37.263056 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 08:24:37.263056 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 08:24:37.263056 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 08:24:37.263056 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:24:37.263056 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 516/1037] 10.222.0.57 () {56 vars in 1120 bytes} [Wed Aug 10 08:24:37 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 08:24:37.268406 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50baa377-e3f4-4d3e-8678-fb66b03ea2ad None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:37.279691 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6580976-bcb2-4372-90de-e7600122c185 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:37.314259 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9b9846-0fe7-4bc8-b9c1-659c86474ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:37.314869 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9b9846-0fe7-4bc8-b9c1-659c86474ab2 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:37.315152 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9b9846-0fe7-4bc8-b9c1-659c86474ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:37.315426 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9b9846-0fe7-4bc8-b9c1-659c86474ab2 None admin] PATH_INFO: `/v3/services` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:37.316053 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9b9846-0fe7-4bc8-b9c1-659c86474ab2 None admin] RBAC: Adding query filter params (name=image) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:37.316363 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9b9846-0fe7-4bc8-b9c1-659c86474ab2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:37.317764 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9b9846-0fe7-4bc8-b9c1-659c86474ab2 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:37.327363 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 522/1038] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:24:37 2022] GET /identity/v3/services?name=image => generated 122 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:24:37.328757 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9b9846-0fe7-4bc8-b9c1-659c86474ab2 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:37.338885 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:37.386412 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50baa377-e3f4-4d3e-8678-fb66b03ea2ad None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:37.422479 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff8586c-0a82-44de-a47b-c0cdd3e49b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:37.423266 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff8586c-0a82-44de-a47b-c0cdd3e49b10 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:37.423557 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff8586c-0a82-44de-a47b-c0cdd3e49b10 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:37.423829 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff8586c-0a82-44de-a47b-c0cdd3e49b10 None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:37.424486 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff8586c-0a82-44de-a47b-c0cdd3e49b10 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:37.424788 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff8586c-0a82-44de-a47b-c0cdd3e49b10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:37.426184 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff8586c-0a82-44de-a47b-c0cdd3e49b10 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:37.434136 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 517/1039] 10.222.0.57 () {56 vars in 1115 bytes} [Wed Aug 10 08:24:37 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:24:37.439780 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff8586c-0a82-44de-a47b-c0cdd3e49b10 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:37.467985 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:37.482125 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9b9846-0fe7-4bc8-b9c1-659c86474ab2 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:37.523202 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371a21e1-54b5-4f6e-85f9-c073b832ef19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:37.524174 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371a21e1-54b5-4f6e-85f9-c073b832ef19 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:37.524455 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371a21e1-54b5-4f6e-85f9-c073b832ef19 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:37.525010 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371a21e1-54b5-4f6e-85f9-c073b832ef19 None admin] PATH_INFO: `/v3/services` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:37.525637 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371a21e1-54b5-4f6e-85f9-c073b832ef19 None admin] RBAC: Adding query filter params (type=image) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:37.526102 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371a21e1-54b5-4f6e-85f9-c073b832ef19 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:37.527763 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371a21e1-54b5-4f6e-85f9-c073b832ef19 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:37.539754 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 523/1040] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:24:37 2022] GET /identity/v3/services?type=image => generated 350 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:24:37.540839 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371a21e1-54b5-4f6e-85f9-c073b832ef19 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:37.543109 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff8586c-0a82-44de-a47b-c0cdd3e49b10 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:37.580744 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34d0091-46d0-42ec-bbc1-6ee09c8889a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:37.581422 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34d0091-46d0-42ec-bbc1-6ee09c8889a6 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:37.581673 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34d0091-46d0-42ec-bbc1-6ee09c8889a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:37.581945 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34d0091-46d0-42ec-bbc1-6ee09c8889a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:37.592422 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34d0091-46d0-42ec-bbc1-6ee09c8889a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:37.593896 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34d0091-46d0-42ec-bbc1-6ee09c8889a6 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:37.608474 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-e34d0091-46d0-42ec-bbc1-6ee09c8889a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:24:37.608474 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:37.608474 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:24:37.608474 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:24:37.608474 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:24:37.608474 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:24:37.608474 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:24:37.608474 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:24:37.608474 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:24:37.608474 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:24:37.608474 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:24:37.608474 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:24:37.608474 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 08:24:37.608474 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 08:24:37.608474 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 08:24:37.608474 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 08:24:37.609561 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:24:37.609561 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:24:37.609561 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:24:37.609561 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:24:37.609561 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:24:37.609561 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:24:37.609561 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:24:37.609561 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:24:37.609561 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:24:37.609561 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:24:37.609561 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:24:37.609561 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:24:37.609561 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:24:37.609561 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:24:37.609561 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:24:37.609561 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:24:37.609561 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 08:24:37.610705 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 08:24:37.610705 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:24:37.610705 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:24:37.610705 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 518/1041] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:24:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 08:24:37.615377 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34d0091-46d0-42ec-bbc1-6ee09c8889a6 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:37.616114 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:37.648382 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371a21e1-54b5-4f6e-85f9-c073b832ef19 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:37.685226 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b19ef50-cd67-4a68-94ba-dfcd73d9342d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:37.685908 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b19ef50-cd67-4a68-94ba-dfcd73d9342d None admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:37.686189 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b19ef50-cd67-4a68-94ba-dfcd73d9342d None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:37.686822 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b19ef50-cd67-4a68-94ba-dfcd73d9342d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:37.687114 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b19ef50-cd67-4a68-94ba-dfcd73d9342d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:37.689194 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b19ef50-cd67-4a68-94ba-dfcd73d9342d None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:37.700285 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:37.722075 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34d0091-46d0-42ec-bbc1-6ee09c8889a6 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:37.756647 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19edae1c-0615-4646-8a94-6d808375c147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:37.757289 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19edae1c-0615-4646-8a94-6d808375c147 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:37.757569 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19edae1c-0615-4646-8a94-6d808375c147 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:37.757839 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19edae1c-0615-4646-8a94-6d808375c147 None admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:37.758487 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19edae1c-0615-4646-8a94-6d808375c147 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:37.758787 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19edae1c-0615-4646-8a94-6d808375c147 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:37.760153 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19edae1c-0615-4646-8a94-6d808375c147 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:37.773282 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 519/1042] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Aug 10 08:24:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:24:37.778950 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19edae1c-0615-4646-8a94-6d808375c147 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:37.812947 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 524/1043] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 08:24:37 2022] POST /identity/v3/endpoints => generated 327 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 08:24:37.889736 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19edae1c-0615-4646-8a94-6d808375c147 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:37.932363 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7bb407-3476-46de-b38c-0544dc611aa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:37.933034 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7bb407-3476-46de-b38c-0544dc611aa9 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:37.933340 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7bb407-3476-46de-b38c-0544dc611aa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:37.933575 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7bb407-3476-46de-b38c-0544dc611aa9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:37.935734 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:37.942932 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7bb407-3476-46de-b38c-0544dc611aa9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:37.944606 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7bb407-3476-46de-b38c-0544dc611aa9 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:37.958111 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-ce7bb407-3476-46de-b38c-0544dc611aa9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 08:24:37.958111 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:37.958111 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:24:37.958111 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:24:37.958111 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:24:37.958111 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:24:37.958111 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:24:37.958111 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:24:37.958111 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:24:37.958111 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:24:37.958111 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:24:37.958111 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:24:37.958111 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 08:24:37.958111 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 08:24:37.958111 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 08:24:37.958111 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 08:24:37.959177 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:24:37.959177 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:24:37.959177 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 08:24:37.959177 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 08:24:37.959177 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 08:24:37.959177 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 08:24:37.959177 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:24:37.959177 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:24:37.959177 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:24:37.959177 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:24:37.959177 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:24:37.959177 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:24:37.959177 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:24:37.959177 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:24:37.959177 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:24:37.959177 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:24:37.959177 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:24:37.959177 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:24:37.960354 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:24:37.960354 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:24:37.960354 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 08:24:37.960354 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 08:24:37.960354 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 08:24:37.960354 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 08:24:37.960354 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 08:24:37.960354 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 08:24:37.960354 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 08:24:37.960354 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:24:37.960354 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 520/1044] 10.222.0.57 () {56 vars in 1093 bytes} [Wed Aug 10 08:24:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 08:24:37.966985 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b19ef50-cd67-4a68-94ba-dfcd73d9342d None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:38.073347 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b19ef50-cd67-4a68-94ba-dfcd73d9342d None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:38.098779 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:38.107035 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385e9092-66a8-424f-b419-9c649c2304c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:38.107685 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385e9092-66a8-424f-b419-9c649c2304c6 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:38.110991 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385e9092-66a8-424f-b419-9c649c2304c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:38.110991 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385e9092-66a8-424f-b419-9c649c2304c6 None admin] PATH_INFO: `/v3/users` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:38.110991 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385e9092-66a8-424f-b419-9c649c2304c6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:38.110991 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385e9092-66a8-424f-b419-9c649c2304c6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:38.111287 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385e9092-66a8-424f-b419-9c649c2304c6 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:38.129600 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 525/1045] 10.222.0.57 () {56 vars in 1133 bytes} [Wed Aug 10 08:24:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:24:38.135983 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7bb407-3476-46de-b38c-0544dc611aa9 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:38.239032 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7bb407-3476-46de-b38c-0544dc611aa9 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:38.272961 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80898343-0df1-485d-8c9c-9f8f07740b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:38.273572 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80898343-0df1-485d-8c9c-9f8f07740b32 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:38.273852 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80898343-0df1-485d-8c9c-9f8f07740b32 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:38.274128 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80898343-0df1-485d-8c9c-9f8f07740b32 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:38.336487 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:38.344314 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80898343-0df1-485d-8c9c-9f8f07740b32 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:38.346045 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80898343-0df1-485d-8c9c-9f8f07740b32 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:38.456700 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 521/1046] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Aug 10 08:24:38 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 10 08:24:38.462204 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385e9092-66a8-424f-b419-9c649c2304c6 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:38.565250 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385e9092-66a8-424f-b419-9c649c2304c6 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:38.599749 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a29b12d-6674-435c-ba47-89c5a4cd813c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:38.600375 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a29b12d-6674-435c-ba47-89c5a4cd813c None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:38.600649 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a29b12d-6674-435c-ba47-89c5a4cd813c None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:38.600924 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a29b12d-6674-435c-ba47-89c5a4cd813c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:38.612162 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a29b12d-6674-435c-ba47-89c5a4cd813c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:38.613625 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a29b12d-6674-435c-ba47-89c5a4cd813c None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:38.629049 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-6a29b12d-6674-435c-ba47-89c5a4cd813c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:24:38.629049 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:38.629049 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:24:38.629049 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:24:38.629049 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:24:38.629049 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:24:38.629049 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:24:38.629049 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:24:38.629049 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:24:38.629049 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:24:38.629049 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:24:38.629049 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:24:38.629049 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 08:24:38.629049 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 08:24:38.629049 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 08:24:38.629049 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 08:24:38.630026 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:24:38.630026 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:24:38.630026 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:24:38.630026 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:24:38.630026 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:24:38.630026 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:24:38.630026 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:24:38.630026 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:24:38.630026 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:24:38.630026 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:24:38.630026 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:24:38.630026 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:24:38.630026 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:24:38.630026 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:24:38.630026 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:24:38.630026 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:24:38.630026 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 08:24:38.630973 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 08:24:38.630973 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:24:38.630973 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:24:38.631954 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 526/1047] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:24:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:24:38.635969 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80898343-0df1-485d-8c9c-9f8f07740b32 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:38.653759 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee6fa83-ae53-46fa-bb4b-2972c59b4e52 None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:38.654035 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee6fa83-ae53-46fa-bb4b-2972c59b4e52 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:38.654304 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee6fa83-ae53-46fa-bb4b-2972c59b4e52 None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:38.655771 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 527/1048] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:24:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:24:38.660637 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812194e5-0dc9-4be3-ba1a-fce8cd2c2fce None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:38.660942 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812194e5-0dc9-4be3-ba1a-fce8cd2c2fce None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:38.661212 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812194e5-0dc9-4be3-ba1a-fce8cd2c2fce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:38.678691 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-812194e5-0dc9-4be3-ba1a-fce8cd2c2fce None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:24:38.679862 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-812194e5-0dc9-4be3-ba1a-fce8cd2c2fce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:24:38.711752 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:38.737183 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:38.744318 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80898343-0df1-485d-8c9c-9f8f07740b32 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:38.752184 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 528/1049] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:24:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 08:24:38.775425 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812194e5-0dc9-4be3-ba1a-fce8cd2c2fce None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:38.780611 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcda913-c921-4297-aa34-fa62b54f54e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:38.781280 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcda913-c921-4297-aa34-fa62b54f54e6 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:38.781593 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcda913-c921-4297-aa34-fa62b54f54e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:38.781860 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcda913-c921-4297-aa34-fa62b54f54e6 None admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:38.782583 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcda913-c921-4297-aa34-fa62b54f54e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:38.782888 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcda913-c921-4297-aa34-fa62b54f54e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:38.784242 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcda913-c921-4297-aa34-fa62b54f54e6 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:38.800980 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 522/1050] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Aug 10 08:24:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:24:38.803024 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcda913-c921-4297-aa34-fa62b54f54e6 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:38.888777 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812194e5-0dc9-4be3-ba1a-fce8cd2c2fce None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:38.919669 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcda913-c921-4297-aa34-fa62b54f54e6 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:38.925511 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac321545-3d0b-4c84-a241-7e173c6bbaa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:38.926071 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac321545-3d0b-4c84-a241-7e173c6bbaa5 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:38.926456 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac321545-3d0b-4c84-a241-7e173c6bbaa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:38.926824 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac321545-3d0b-4c84-a241-7e173c6bbaa5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:38.940104 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac321545-3d0b-4c84-a241-7e173c6bbaa5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 08:24:38.944759 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-ac321545-3d0b-4c84-a241-7e173c6bbaa5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 08:24:38.944759 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:38.944759 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:24:38.944759 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:24:38.944759 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:24:38.944759 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:24:38.944759 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:24:38.944759 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:24:38.944759 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:24:38.944759 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:24:38.944759 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:24:38.944759 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:24:38.944759 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 08:24:38.944759 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 08:24:38.944759 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 08:24:38.944759 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 08:24:38.945759 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 08:24:38.945759 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 08:24:38.945759 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 08:24:38.945759 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 08:24:38.945759 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:24:38.945759 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:24:38.945759 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:24:38.945759 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:24:38.945759 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:24:38.945759 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:24:38.945759 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:24:38.945759 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:24:38.945759 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:24:38.945759 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:24:38.945759 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:24:38.945759 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:24:38.950083 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:24:38.950083 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:24:38.950083 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:24:38.950083 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:24:38.950083 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 08:24:38.950083 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 08:24:38.950083 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 08:24:38.950083 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 08:24:38.950083 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 08:24:38.950083 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 08:24:38.950083 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 08:24:38.950083 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:24:38.950083 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 529/1051] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Aug 10 08:24:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 08:24:38.952146 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac321545-3d0b-4c84-a241-7e173c6bbaa5 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:38.964052 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615a8d8c-1672-48f7-a611-749ace4b5655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:38.964708 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615a8d8c-1672-48f7-a611-749ace4b5655 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:38.965022 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615a8d8c-1672-48f7-a611-749ace4b5655 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:38.965298 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615a8d8c-1672-48f7-a611-749ace4b5655 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:38.975176 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615a8d8c-1672-48f7-a611-749ace4b5655 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:38.976641 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615a8d8c-1672-48f7-a611-749ace4b5655 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:38.983284 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:38.989031 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-615a8d8c-1672-48f7-a611-749ace4b5655 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 08:24:38.989031 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:38.989031 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:24:38.989031 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:24:38.989031 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:24:38.989031 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:24:38.989031 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:24:38.989031 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:24:38.989031 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:24:38.989031 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:24:38.989031 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:24:38.989031 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:24:38.989031 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 08:24:38.989031 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 08:24:38.989031 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 08:24:38.990135 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 08:24:38.990135 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:24:38.990135 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:24:38.990135 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:24:38.990135 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:24:38.990135 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:24:38.990135 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:24:38.990135 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:24:38.990135 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:24:38.990135 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:24:38.990135 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:24:38.990135 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:24:38.990135 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:24:38.990135 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:24:38.990135 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:24:38.990135 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:24:38.990135 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:24:38.991682 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 08:24:38.991682 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 08:24:38.991682 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 08:24:38.991682 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 08:24:38.991682 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 08:24:38.991682 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 08:24:38.991682 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 08:24:38.991682 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:24:38.991682 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 523/1052] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Aug 10 08:24:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 08:24:38.996009 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615a8d8c-1672-48f7-a611-749ace4b5655 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:39.014041 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:39.062806 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac321545-3d0b-4c84-a241-7e173c6bbaa5 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:39.101557 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137b59e0-50d4-4f02-8a6f-63078f28600b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:39.102383 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137b59e0-50d4-4f02-8a6f-63078f28600b None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:39.102810 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137b59e0-50d4-4f02-8a6f-63078f28600b None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:39.103264 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137b59e0-50d4-4f02-8a6f-63078f28600b None admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:39.104079 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137b59e0-50d4-4f02-8a6f-63078f28600b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:39.104530 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137b59e0-50d4-4f02-8a6f-63078f28600b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:39.104663 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615a8d8c-1672-48f7-a611-749ace4b5655 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:39.106299 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137b59e0-50d4-4f02-8a6f-63078f28600b None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:39.120501 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 530/1053] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Aug 10 08:24:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:24:39.122023 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137b59e0-50d4-4f02-8a6f-63078f28600b None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:39.143212 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701755ae-9c8a-4479-82ff-be3c42e5821a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:39.143781 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701755ae-9c8a-4479-82ff-be3c42e5821a None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:39.144026 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701755ae-9c8a-4479-82ff-be3c42e5821a None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:39.144567 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701755ae-9c8a-4479-82ff-be3c42e5821a None admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:39.145014 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701755ae-9c8a-4479-82ff-be3c42e5821a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:39.145321 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701755ae-9c8a-4479-82ff-be3c42e5821a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:39.146772 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701755ae-9c8a-4479-82ff-be3c42e5821a None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:39.159780 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 524/1054] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Aug 10 08:24:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:24:39.165489 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701755ae-9c8a-4479-82ff-be3c42e5821a None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:39.213585 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:39.226548 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137b59e0-50d4-4f02-8a6f-63078f28600b None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:39.247494 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:39.267304 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920e400e-4879-4618-b077-b9982ed24070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:39.268032 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920e400e-4879-4618-b077-b9982ed24070 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:39.268391 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920e400e-4879-4618-b077-b9982ed24070 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:39.268776 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920e400e-4879-4618-b077-b9982ed24070 None admin] PATH_INFO: `/v3/users/5e225fe0b5ba404e84e818b9e3bb40bf` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:39.273753 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701755ae-9c8a-4479-82ff-be3c42e5821a None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:39.284845 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920e400e-4879-4618-b077-b9982ed24070 None admin] RBAC: Authorizing `identity:get_user(user_id=5e225fe0b5ba404e84e818b9e3bb40bf)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:39.286655 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920e400e-4879-4618-b077-b9982ed24070 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:39.288690 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 531/1055] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Aug 10 08:24:39 2022] GET /identity/v3/users/5e225fe0b5ba404e84e818b9e3bb40bf => generated 285 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:39.294061 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920e400e-4879-4618-b077-b9982ed24070 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:39.316732 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e42914c-74bc-4b8c-8d22-6b9c7223cbf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:39.317792 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e42914c-74bc-4b8c-8d22-6b9c7223cbf8 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:39.317792 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e42914c-74bc-4b8c-8d22-6b9c7223cbf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:39.318027 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e42914c-74bc-4b8c-8d22-6b9c7223cbf8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:39.318707 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e42914c-74bc-4b8c-8d22-6b9c7223cbf8 None admin] RBAC: Adding query filter params (role.id=c30407dbef394307b4d8d3dc9ca4bcaa, scope.project.id=b49e381419e940278bcb6111899589b5, user.id=a943008193914f76a663d489bd73e873) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:39.319014 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e42914c-74bc-4b8c-8d22-6b9c7223cbf8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:39.320420 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e42914c-74bc-4b8c-8d22-6b9c7223cbf8 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:39.336946 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 525/1056] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Aug 10 08:24:39 2022] GET /identity/v3/role_assignments?user.id=a943008193914f76a663d489bd73e873&scope.project.id=b49e381419e940278bcb6111899589b5&role.id=c30407dbef394307b4d8d3dc9ca4bcaa => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:24:39.420401 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920e400e-4879-4618-b077-b9982ed24070 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:39.457992 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2e2026-ab02-4c79-9d6a-625aebcd4f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:39.458626 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2e2026-ab02-4c79-9d6a-625aebcd4f30 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:39.458907 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2e2026-ab02-4c79-9d6a-625aebcd4f30 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:39.459179 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2e2026-ab02-4c79-9d6a-625aebcd4f30 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:39.476778 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:39.535600 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2e2026-ab02-4c79-9d6a-625aebcd4f30 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:39.537088 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2e2026-ab02-4c79-9d6a-625aebcd4f30 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:39.644937 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 532/1057] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Aug 10 08:24:39 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 10 08:24:39.646583 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e42914c-74bc-4b8c-8d22-6b9c7223cbf8 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:39.748212 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e42914c-74bc-4b8c-8d22-6b9c7223cbf8 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:39.784298 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad12e9f-3d9c-431b-b9c6-98aa486fe1a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:39.784298 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad12e9f-3d9c-431b-b9c6-98aa486fe1a2 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:39.784298 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad12e9f-3d9c-431b-b9c6-98aa486fe1a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:39.784653 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad12e9f-3d9c-431b-b9c6-98aa486fe1a2 None admin] PATH_INFO: `/v3/projects/e41afb9d169145b0abf42179e2a02bdf` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:39.785588 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad12e9f-3d9c-431b-b9c6-98aa486fe1a2 None admin] RBAC: Authorizing `identity:get_project(project_id=e41afb9d169145b0abf42179e2a02bdf)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:39.787007 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad12e9f-3d9c-431b-b9c6-98aa486fe1a2 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:39.792975 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 526/1058] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Aug 10 08:24:39 2022] GET /identity/v3/projects/e41afb9d169145b0abf42179e2a02bdf => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:24:39.795188 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2e2026-ab02-4c79-9d6a-625aebcd4f30 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:39.842656 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:39.881580 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5031123c-6d3c-4751-ae56-4d269c0f60af None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:39.881862 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5031123c-6d3c-4751-ae56-4d269c0f60af None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:39.882129 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5031123c-6d3c-4751-ae56-4d269c0f60af None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:39.883599 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 527/1059] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:24:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 08:24:39.888123 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791d92d1-bab3-40df-99d1-ce9a21ccad4c None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:39.888401 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791d92d1-bab3-40df-99d1-ce9a21ccad4c None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:39.888669 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791d92d1-bab3-40df-99d1-ce9a21ccad4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:39.899016 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2e2026-ab02-4c79-9d6a-625aebcd4f30 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:39.902279 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:39.909019 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-791d92d1-bab3-40df-99d1-ce9a21ccad4c None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:24:39.910656 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-791d92d1-bab3-40df-99d1-ce9a21ccad4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:24:39.935840 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13cc4df-4b0a-4a5e-93d7-64d759140d81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:39.936553 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13cc4df-4b0a-4a5e-93d7-64d759140d81 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:39.936827 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13cc4df-4b0a-4a5e-93d7-64d759140d81 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:39.937237 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13cc4df-4b0a-4a5e-93d7-64d759140d81 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:39.937954 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13cc4df-4b0a-4a5e-93d7-64d759140d81 None admin] RBAC: Adding query filter params (role.id=ec3751692d9c4ecf81b4a71c936c46be, scope.project.id=e41afb9d169145b0abf42179e2a02bdf, user.id=5e225fe0b5ba404e84e818b9e3bb40bf) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:39.938851 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13cc4df-4b0a-4a5e-93d7-64d759140d81 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:39.940404 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13cc4df-4b0a-4a5e-93d7-64d759140d81 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:39.956948 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 533/1060] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Aug 10 08:24:39 2022] GET /identity/v3/role_assignments?user.id=5e225fe0b5ba404e84e818b9e3bb40bf&scope.project.id=e41afb9d169145b0abf42179e2a02bdf&role.id=ec3751692d9c4ecf81b4a71c936c46be => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:39.995239 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 528/1061] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:24:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 08:24:40.019239 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13cc4df-4b0a-4a5e-93d7-64d759140d81 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:40.130352 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13cc4df-4b0a-4a5e-93d7-64d759140d81 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:40.141183 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:40.165034 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41725766-c8ad-4fcf-9661-ef458bb245ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:40.165639 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41725766-c8ad-4fcf-9661-ef458bb245ea None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:40.165920 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41725766-c8ad-4fcf-9661-ef458bb245ea None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:40.166188 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41725766-c8ad-4fcf-9661-ef458bb245ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:40.233900 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41725766-c8ad-4fcf-9661-ef458bb245ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:40.235617 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41725766-c8ad-4fcf-9661-ef458bb245ea None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:40.338986 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 534/1062] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Aug 10 08:24:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 08:24:40.340158 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791d92d1-bab3-40df-99d1-ce9a21ccad4c None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:40.464490 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791d92d1-bab3-40df-99d1-ce9a21ccad4c None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:40.499319 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4953b54a-61fb-4753-96fa-f88e26bf6700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:40.499942 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4953b54a-61fb-4753-96fa-f88e26bf6700 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:40.500215 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4953b54a-61fb-4753-96fa-f88e26bf6700 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:40.500483 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4953b54a-61fb-4753-96fa-f88e26bf6700 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:40.503296 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:40.511284 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4953b54a-61fb-4753-96fa-f88e26bf6700 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:40.512695 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4953b54a-61fb-4753-96fa-f88e26bf6700 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:40.527119 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-4953b54a-61fb-4753-96fa-f88e26bf6700 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:24:40.527119 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:40.527119 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:24:40.527119 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:24:40.527119 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:24:40.527119 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:24:40.527119 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:24:40.527119 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:24:40.527119 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:24:40.527119 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:24:40.527119 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:24:40.527119 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:24:40.527119 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 08:24:40.527119 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 08:24:40.527119 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 08:24:40.527119 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 08:24:40.528073 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:24:40.528073 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:24:40.528073 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:24:40.528073 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:24:40.528073 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:24:40.528073 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:24:40.528073 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:24:40.528073 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:24:40.528073 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:24:40.528073 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:24:40.528073 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:24:40.528073 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:24:40.528073 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:24:40.528073 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:24:40.528073 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:24:40.528073 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:24:40.528073 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 08:24:40.529620 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 08:24:40.529620 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:24:40.529620 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:24:40.529620 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 529/1063] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:24:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:24:40.535032 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41725766-c8ad-4fcf-9661-ef458bb245ea None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:40.546612 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:40.635129 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41725766-c8ad-4fcf-9661-ef458bb245ea None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:40.668945 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107099c5-a1b7-49fd-ad8a-321364b67218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:40.669551 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107099c5-a1b7-49fd-ad8a-321364b67218 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:40.669830 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107099c5-a1b7-49fd-ad8a-321364b67218 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:40.670096 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107099c5-a1b7-49fd-ad8a-321364b67218 None admin] PATH_INFO: `/v3/domains` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:40.670728 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107099c5-a1b7-49fd-ad8a-321364b67218 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:40.671029 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107099c5-a1b7-49fd-ad8a-321364b67218 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:40.672385 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107099c5-a1b7-49fd-ad8a-321364b67218 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:40.692966 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 535/1064] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Aug 10 08:24:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:41.340358 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3059749b-42fc-4247-a9f9-65c3c7ac3a28 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:41.340982 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3059749b-42fc-4247-a9f9-65c3c7ac3a28 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:41.340982 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3059749b-42fc-4247-a9f9-65c3c7ac3a28 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:41.342277 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 530/1065] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:24:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:24:41.347449 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ff1463-ba72-474d-a16b-aa0284f0a796 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:41.347838 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ff1463-ba72-474d-a16b-aa0284f0a796 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:41.347990 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ff1463-ba72-474d-a16b-aa0284f0a796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:41.375165 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-d9ff1463-ba72-474d-a16b-aa0284f0a796 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:24:41.377005 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-d9ff1463-ba72-474d-a16b-aa0284f0a796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:24:41.458854 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:41.463276 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 536/1066] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:24:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 08:24:41.487397 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3059749b-42fc-4247-a9f9-65c3c7ac3a28 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:41.587977 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3059749b-42fc-4247-a9f9-65c3c7ac3a28 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:41.629795 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586c13e4-4b81-4085-ba10-db1e98a999e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:41.629795 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586c13e4-4b81-4085-ba10-db1e98a999e4 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:41.629795 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586c13e4-4b81-4085-ba10-db1e98a999e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:41.629795 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586c13e4-4b81-4085-ba10-db1e98a999e4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:41.634428 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586c13e4-4b81-4085-ba10-db1e98a999e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 08:24:41.639263 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-586c13e4-4b81-4085-ba10-db1e98a999e4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 08:24:41.639263 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:41.639263 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:24:41.639263 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:24:41.639263 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:24:41.639263 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:24:41.639263 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:24:41.639263 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:24:41.639263 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:24:41.639263 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:24:41.639263 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:24:41.639263 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:24:41.639263 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 08:24:41.639263 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 08:24:41.639263 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 08:24:41.639263 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 08:24:41.645367 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 08:24:41.645367 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 08:24:41.645367 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 08:24:41.645367 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 08:24:41.645367 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:24:41.645367 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:24:41.645367 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:24:41.645367 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:24:41.645367 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:24:41.645367 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:24:41.645367 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:24:41.645367 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:24:41.645367 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:24:41.645367 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:24:41.645367 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:24:41.645367 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:24:41.646504 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:24:41.646504 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:24:41.646504 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:24:41.646504 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:24:41.646504 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 08:24:41.646504 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 08:24:41.646504 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 08:24:41.646504 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 08:24:41.646504 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 08:24:41.646504 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 08:24:41.646504 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 08:24:41.646504 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:24:41.646504 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 531/1067] 10.222.0.57 () {56 vars in 1120 bytes} [Wed Aug 10 08:24:41 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:24:41.648435 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ff1463-ba72-474d-a16b-aa0284f0a796 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:41.747555 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ff1463-ba72-474d-a16b-aa0284f0a796 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:41.780063 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708d9c1d-d2da-4ff8-b99f-05bd6a289431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:41.780665 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708d9c1d-d2da-4ff8-b99f-05bd6a289431 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:41.780963 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708d9c1d-d2da-4ff8-b99f-05bd6a289431 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:41.781239 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708d9c1d-d2da-4ff8-b99f-05bd6a289431 None admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:41.782446 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708d9c1d-d2da-4ff8-b99f-05bd6a289431 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:41.782878 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708d9c1d-d2da-4ff8-b99f-05bd6a289431 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:41.784238 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708d9c1d-d2da-4ff8-b99f-05bd6a289431 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:41.791933 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 537/1068] 10.222.0.57 () {56 vars in 1115 bytes} [Wed Aug 10 08:24:41 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:41.798424 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586c13e4-4b81-4085-ba10-db1e98a999e4 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:41.846967 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:41.897701 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586c13e4-4b81-4085-ba10-db1e98a999e4 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:41.931424 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e61b5b-9c59-41f9-8536-b8e7294a6237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:41.932042 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e61b5b-9c59-41f9-8536-b8e7294a6237 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:41.932375 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e61b5b-9c59-41f9-8536-b8e7294a6237 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:41.932650 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e61b5b-9c59-41f9-8536-b8e7294a6237 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:41.943138 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e61b5b-9c59-41f9-8536-b8e7294a6237 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:41.944745 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e61b5b-9c59-41f9-8536-b8e7294a6237 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:41.945237 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a58d8b-79e0-42cd-93e6-c4161c6a6881 None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:41.945527 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a58d8b-79e0-42cd-93e6-c4161c6a6881 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:41.945805 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a58d8b-79e0-42cd-93e6-c4161c6a6881 None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:41.947281 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 538/1069] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:24:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 08:24:41.951926 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24982e0f-94c9-44b2-9b21-ffad3bfdfaab None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:41.952271 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24982e0f-94c9-44b2-9b21-ffad3bfdfaab None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:41.952611 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24982e0f-94c9-44b2-9b21-ffad3bfdfaab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:41.959373 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-36e61b5b-9c59-41f9-8536-b8e7294a6237 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:24:41.959373 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:41.959373 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:24:41.959373 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:24:41.959373 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:24:41.959373 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:24:41.959373 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:24:41.959373 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:24:41.959373 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:24:41.959373 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:24:41.959373 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:24:41.959373 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:24:41.959373 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 08:24:41.959373 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 08:24:41.959373 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 08:24:41.959373 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 08:24:41.960403 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:24:41.960403 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:24:41.960403 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:24:41.960403 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:24:41.960403 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:24:41.960403 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:24:41.960403 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:24:41.960403 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:24:41.960403 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:24:41.960403 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:24:41.960403 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:24:41.960403 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:24:41.960403 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:24:41.960403 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:24:41.960403 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:24:41.960403 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:24:41.960403 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 08:24:41.961389 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 08:24:41.961389 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:24:41.961389 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:24:41.961389 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 532/1070] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:24:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:24:41.966150 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e61b5b-9c59-41f9-8536-b8e7294a6237 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:41.973052 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-24982e0f-94c9-44b2-9b21-ffad3bfdfaab None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:24:41.974241 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-24982e0f-94c9-44b2-9b21-ffad3bfdfaab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:24:42.029653 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:42.040363 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 539/1071] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:24:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 08:24:42.060970 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:42.062698 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24982e0f-94c9-44b2-9b21-ffad3bfdfaab None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:42.064592 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e61b5b-9c59-41f9-8536-b8e7294a6237 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:42.098306 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c90963-e8b4-46f7-acfe-318f531e3013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:42.098905 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c90963-e8b4-46f7-acfe-318f531e3013 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:42.099182 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c90963-e8b4-46f7-acfe-318f531e3013 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:42.099452 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c90963-e8b4-46f7-acfe-318f531e3013 None admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:42.100093 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c90963-e8b4-46f7-acfe-318f531e3013 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:42.100394 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c90963-e8b4-46f7-acfe-318f531e3013 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:42.102119 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c90963-e8b4-46f7-acfe-318f531e3013 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:42.113732 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 533/1072] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Aug 10 08:24:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:24:42.119614 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c90963-e8b4-46f7-acfe-318f531e3013 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:42.160488 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24982e0f-94c9-44b2-9b21-ffad3bfdfaab None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:42.193729 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7290bd36-5b32-41d9-bfb3-9613369c8f7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:42.194378 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7290bd36-5b32-41d9-bfb3-9613369c8f7e None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:42.194662 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7290bd36-5b32-41d9-bfb3-9613369c8f7e None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:42.194930 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7290bd36-5b32-41d9-bfb3-9613369c8f7e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:42.205639 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7290bd36-5b32-41d9-bfb3-9613369c8f7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 08:24:42.210422 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-7290bd36-5b32-41d9-bfb3-9613369c8f7e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 08:24:42.210422 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:42.210422 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:24:42.210422 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:24:42.210422 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:24:42.210422 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:24:42.210422 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:24:42.210422 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:24:42.210422 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:24:42.210422 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:24:42.210422 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:24:42.210422 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:24:42.210422 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 08:24:42.210422 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 08:24:42.210422 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 08:24:42.210422 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 08:24:42.211426 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 08:24:42.211426 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 08:24:42.211426 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 08:24:42.211426 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 08:24:42.211426 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:24:42.211426 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:24:42.211426 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:24:42.211426 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:24:42.211426 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:24:42.211426 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:24:42.211426 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:24:42.211426 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:24:42.211426 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:24:42.211426 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:24:42.211426 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:24:42.211426 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:24:42.212285 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:24:42.212285 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:24:42.212285 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:24:42.212285 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:24:42.212285 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 08:24:42.212285 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 08:24:42.212285 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 08:24:42.212285 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 08:24:42.212285 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 08:24:42.212285 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 08:24:42.212285 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 08:24:42.212285 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:24:42.212285 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 540/1073] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Aug 10 08:24:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 08:24:42.217476 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7290bd36-5b32-41d9-bfb3-9613369c8f7e None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:42.220331 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c90963-e8b4-46f7-acfe-318f531e3013 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:42.255581 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16958480-9cef-41ac-8a33-8a7fc04241a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:42.256201 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16958480-9cef-41ac-8a33-8a7fc04241a2 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:42.256480 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16958480-9cef-41ac-8a33-8a7fc04241a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:42.256749 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16958480-9cef-41ac-8a33-8a7fc04241a2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:42.263733 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16958480-9cef-41ac-8a33-8a7fc04241a2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:42.265251 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16958480-9cef-41ac-8a33-8a7fc04241a2 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:42.272308 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:42.277162 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-16958480-9cef-41ac-8a33-8a7fc04241a2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 08:24:42.277162 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:42.277162 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:24:42.277162 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:24:42.277162 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:24:42.277162 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:24:42.277162 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:24:42.277162 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:24:42.277162 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:24:42.277162 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:24:42.277162 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:24:42.277162 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:24:42.277162 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 08:24:42.277162 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 08:24:42.277162 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 08:24:42.277162 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 08:24:42.278183 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:24:42.278183 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:24:42.278183 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 08:24:42.278183 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 08:24:42.278183 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 08:24:42.278183 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 08:24:42.278183 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:24:42.278183 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:24:42.278183 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:24:42.278183 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:24:42.278183 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:24:42.278183 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:24:42.278183 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:24:42.278183 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:24:42.278183 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:24:42.278183 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:24:42.278183 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:24:42.278183 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:24:42.279216 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:24:42.279216 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:24:42.279216 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 08:24:42.279216 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 08:24:42.279216 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 08:24:42.279216 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 08:24:42.279216 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 08:24:42.279216 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 08:24:42.279216 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 08:24:42.279216 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:24:42.279216 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 534/1074] 10.222.0.57 () {56 vars in 1093 bytes} [Wed Aug 10 08:24:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 08:24:42.284069 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16958480-9cef-41ac-8a33-8a7fc04241a2 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:42.305226 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:42.321096 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7290bd36-5b32-41d9-bfb3-9613369c8f7e None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:42.355328 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d3776a-d166-4f1c-a663-9723c9e3069b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:42.355969 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d3776a-d166-4f1c-a663-9723c9e3069b None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:42.356190 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d3776a-d166-4f1c-a663-9723c9e3069b None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:42.356455 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d3776a-d166-4f1c-a663-9723c9e3069b None admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:42.357148 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d3776a-d166-4f1c-a663-9723c9e3069b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:42.357447 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d3776a-d166-4f1c-a663-9723c9e3069b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:42.358799 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d3776a-d166-4f1c-a663-9723c9e3069b None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:42.366981 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 541/1075] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Aug 10 08:24:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:24:42.372999 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d3776a-d166-4f1c-a663-9723c9e3069b None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:42.383041 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16958480-9cef-41ac-8a33-8a7fc04241a2 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:42.418979 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc47660-86b7-42c9-aa6d-6d0898b942d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:42.419515 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc47660-86b7-42c9-aa6d-6d0898b942d4 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:42.419792 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc47660-86b7-42c9-aa6d-6d0898b942d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:42.420071 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc47660-86b7-42c9-aa6d-6d0898b942d4 None admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:42.420798 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc47660-86b7-42c9-aa6d-6d0898b942d4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:42.421179 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc47660-86b7-42c9-aa6d-6d0898b942d4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:42.422619 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc47660-86b7-42c9-aa6d-6d0898b942d4 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:42.440166 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 535/1076] 10.222.0.57 () {56 vars in 1133 bytes} [Wed Aug 10 08:24:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:24:42.445862 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc47660-86b7-42c9-aa6d-6d0898b942d4 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:42.475652 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d3776a-d166-4f1c-a663-9723c9e3069b None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:42.489057 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:42.509283 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c399a568-e589-4a91-9e6f-462275e30043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:42.509910 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c399a568-e589-4a91-9e6f-462275e30043 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:42.510219 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c399a568-e589-4a91-9e6f-462275e30043 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:42.510490 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c399a568-e589-4a91-9e6f-462275e30043 None admin] PATH_INFO: `/v3/users/5e225fe0b5ba404e84e818b9e3bb40bf` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:42.511649 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c399a568-e589-4a91-9e6f-462275e30043 None admin] RBAC: Authorizing `identity:get_user(user_id=5e225fe0b5ba404e84e818b9e3bb40bf)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:42.513175 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c399a568-e589-4a91-9e6f-462275e30043 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:42.514754 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 542/1077] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Aug 10 08:24:42 2022] GET /identity/v3/users/5e225fe0b5ba404e84e818b9e3bb40bf => generated 285 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:24:42.521047 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c399a568-e589-4a91-9e6f-462275e30043 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:42.522373 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:42.543229 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc47660-86b7-42c9-aa6d-6d0898b942d4 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:42.578310 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8ef79d-e351-4834-9cf7-d05299c4ec68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:42.578949 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8ef79d-e351-4834-9cf7-d05299c4ec68 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:42.579219 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8ef79d-e351-4834-9cf7-d05299c4ec68 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:42.579485 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8ef79d-e351-4834-9cf7-d05299c4ec68 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:42.589122 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8ef79d-e351-4834-9cf7-d05299c4ec68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:42.590552 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8ef79d-e351-4834-9cf7-d05299c4ec68 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:42.604432 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-7e8ef79d-e351-4834-9cf7-d05299c4ec68 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:24:42.604432 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:42.604432 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:24:42.604432 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:24:42.604432 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:24:42.604432 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:24:42.604432 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:24:42.604432 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:24:42.604432 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:24:42.604432 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:24:42.604432 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:24:42.604432 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:24:42.604432 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 08:24:42.604432 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 08:24:42.604432 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 08:24:42.604432 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 08:24:42.605698 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:24:42.605698 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:24:42.605698 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:24:42.605698 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:24:42.605698 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:24:42.605698 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:24:42.605698 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:24:42.605698 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:24:42.605698 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:24:42.605698 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:24:42.605698 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:24:42.605698 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:24:42.605698 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:24:42.605698 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:24:42.605698 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:24:42.605698 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:24:42.605698 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 08:24:42.606864 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 08:24:42.606864 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:24:42.606864 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:24:42.606864 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 536/1078] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:24:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 08:24:42.611868 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8ef79d-e351-4834-9cf7-d05299c4ec68 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:42.622032 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c399a568-e589-4a91-9e6f-462275e30043 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:42.655885 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6718f858-f630-4670-ba13-98f30837fb22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:42.656502 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6718f858-f630-4670-ba13-98f30837fb22 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:42.656802 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6718f858-f630-4670-ba13-98f30837fb22 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:42.657098 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6718f858-f630-4670-ba13-98f30837fb22 None admin] PATH_INFO: `/v3/projects/e41afb9d169145b0abf42179e2a02bdf` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:42.658203 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6718f858-f630-4670-ba13-98f30837fb22 None admin] RBAC: Authorizing `identity:get_project(project_id=e41afb9d169145b0abf42179e2a02bdf)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:42.659623 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6718f858-f630-4670-ba13-98f30837fb22 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:42.661288 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 543/1079] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Aug 10 08:24:42 2022] GET /identity/v3/projects/e41afb9d169145b0abf42179e2a02bdf => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:42.667155 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6718f858-f630-4670-ba13-98f30837fb22 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:42.709627 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8ef79d-e351-4834-9cf7-d05299c4ec68 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:42.717161 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:42.738661 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:42.744806 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac6427e-b283-45a2-9030-a2d7ba38f796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:42.745450 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac6427e-b283-45a2-9030-a2d7ba38f796 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:42.745734 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac6427e-b283-45a2-9030-a2d7ba38f796 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:42.746005 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac6427e-b283-45a2-9030-a2d7ba38f796 None admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:42.746775 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac6427e-b283-45a2-9030-a2d7ba38f796 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:42.747007 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac6427e-b283-45a2-9030-a2d7ba38f796 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:42.748381 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac6427e-b283-45a2-9030-a2d7ba38f796 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:42.760696 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 537/1080] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Aug 10 08:24:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:24:42.766375 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac6427e-b283-45a2-9030-a2d7ba38f796 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:42.769651 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6718f858-f630-4670-ba13-98f30837fb22 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:42.804163 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83030e1-2744-4342-be77-899a12b6549b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:42.804816 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83030e1-2744-4342-be77-899a12b6549b None admin] REQUEST_METHOD: `PUT` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:42.805133 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83030e1-2744-4342-be77-899a12b6549b None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:42.805410 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83030e1-2744-4342-be77-899a12b6549b None admin] PATH_INFO: `/v3/projects/e41afb9d169145b0abf42179e2a02bdf/users/5e225fe0b5ba404e84e818b9e3bb40bf/roles/ec3751692d9c4ecf81b4a71c936c46be` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:42.807314 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83030e1-2744-4342-be77-899a12b6549b None admin] RBAC: Authorizing `identity:create_grant(project_id=e41afb9d169145b0abf42179e2a02bdf, user_id=5e225fe0b5ba404e84e818b9e3bb40bf, role_id=ec3751692d9c4ecf81b4a71c936c46be)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:42.808746 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83030e1-2744-4342-be77-899a12b6549b None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:42.863452 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac6427e-b283-45a2-9030-a2d7ba38f796 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:42.898101 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4570d335-abd8-40c0-ae97-deb7476a8af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:42.898716 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4570d335-abd8-40c0-ae97-deb7476a8af7 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:42.898990 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4570d335-abd8-40c0-ae97-deb7476a8af7 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:42.899258 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4570d335-abd8-40c0-ae97-deb7476a8af7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:42.905714 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4570d335-abd8-40c0-ae97-deb7476a8af7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:42.907621 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4570d335-abd8-40c0-ae97-deb7476a8af7 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:42.908299 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 544/1081] 10.222.0.57 () {58 vars in 1439 bytes} [Wed Aug 10 08:24:42 2022] PUT /identity/v3/projects/e41afb9d169145b0abf42179e2a02bdf/users/5e225fe0b5ba404e84e818b9e3bb40bf/roles/ec3751692d9c4ecf81b4a71c936c46be => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 08:24:42.918817 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-4570d335-abd8-40c0-ae97-deb7476a8af7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 08:24:42.918817 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:42.918817 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:24:42.918817 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:24:42.918817 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:24:42.918817 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:24:42.918817 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:24:42.918817 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:24:42.918817 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:24:42.918817 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:24:42.918817 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:24:42.918817 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:24:42.918817 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 08:24:42.918817 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 08:24:42.918817 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 08:24:42.919996 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 08:24:42.919996 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:24:42.919996 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:24:42.919996 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:24:42.919996 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:24:42.919996 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:24:42.919996 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:24:42.919996 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:24:42.919996 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:24:42.919996 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:24:42.919996 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:24:42.919996 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:24:42.919996 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:24:42.919996 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:24:42.919996 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:24:42.919996 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:24:42.919996 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:24:42.921146 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 08:24:42.921146 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 08:24:42.921146 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 08:24:42.921146 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 08:24:42.921146 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 08:24:42.921146 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 08:24:42.921146 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 08:24:42.921146 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:24:42.921146 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 538/1082] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Aug 10 08:24:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 08:24:42.926890 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83030e1-2744-4342-be77-899a12b6549b None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:43.030661 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83030e1-2744-4342-be77-899a12b6549b None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:43.042148 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:43.066618 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab7c5e0-21f0-4ca3-a701-8b5b8ab4456a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:43.066618 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab7c5e0-21f0-4ca3-a701-8b5b8ab4456a None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:43.066953 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab7c5e0-21f0-4ca3-a701-8b5b8ab4456a None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:43.066953 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab7c5e0-21f0-4ca3-a701-8b5b8ab4456a None admin] PATH_INFO: `/v3/projects` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:43.068158 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab7c5e0-21f0-4ca3-a701-8b5b8ab4456a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:43.068158 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab7c5e0-21f0-4ca3-a701-8b5b8ab4456a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:43.069328 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab7c5e0-21f0-4ca3-a701-8b5b8ab4456a None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:43.082087 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 545/1083] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Aug 10 08:24:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:43.088050 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4570d335-abd8-40c0-ae97-deb7476a8af7 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:43.139363 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:43.189018 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4570d335-abd8-40c0-ae97-deb7476a8af7 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:43.222242 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92d295e-6d2d-4937-a73b-45fac200c4f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:43.222898 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92d295e-6d2d-4937-a73b-45fac200c4f6 None admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:43.223179 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92d295e-6d2d-4937-a73b-45fac200c4f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:43.223451 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92d295e-6d2d-4937-a73b-45fac200c4f6 None admin] PATH_INFO: `/v3/projects/b49e381419e940278bcb6111899589b5/users/a943008193914f76a663d489bd73e873/roles/c30407dbef394307b4d8d3dc9ca4bcaa` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:43.225580 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92d295e-6d2d-4937-a73b-45fac200c4f6 None admin] RBAC: Authorizing `identity:create_grant(project_id=b49e381419e940278bcb6111899589b5, user_id=a943008193914f76a663d489bd73e873, role_id=c30407dbef394307b4d8d3dc9ca4bcaa)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:43.226996 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92d295e-6d2d-4937-a73b-45fac200c4f6 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:43.296992 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 539/1084] 10.222.0.57 () {58 vars in 1439 bytes} [Wed Aug 10 08:24:43 2022] PUT /identity/v3/projects/b49e381419e940278bcb6111899589b5/users/a943008193914f76a663d489bd73e873/roles/c30407dbef394307b4d8d3dc9ca4bcaa => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:24:44.907547 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92e9ad1-6255-4a93-890f-fc2cb58d651c None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:44.908116 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92e9ad1-6255-4a93-890f-fc2cb58d651c None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:44.908116 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92e9ad1-6255-4a93-890f-fc2cb58d651c None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:44.910049 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 546/1085] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:24:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:24:44.914707 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32a24cd-e9c2-4b54-920a-f2263f0cb48c None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:44.915131 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32a24cd-e9c2-4b54-920a-f2263f0cb48c None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:44.915251 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32a24cd-e9c2-4b54-920a-f2263f0cb48c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:44.932952 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-a32a24cd-e9c2-4b54-920a-f2263f0cb48c None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:24:44.934133 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-a32a24cd-e9c2-4b54-920a-f2263f0cb48c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:24:45.000833 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 540/1086] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:24:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 08:24:45.024093 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92e9ad1-6255-4a93-890f-fc2cb58d651c None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:45.123234 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92e9ad1-6255-4a93-890f-fc2cb58d651c None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:45.156235 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598e8b5a-0717-4d84-9085-ec612d1bd5da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:45.157426 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598e8b5a-0717-4d84-9085-ec612d1bd5da None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:45.157786 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598e8b5a-0717-4d84-9085-ec612d1bd5da None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:45.157969 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598e8b5a-0717-4d84-9085-ec612d1bd5da None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:45.167795 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598e8b5a-0717-4d84-9085-ec612d1bd5da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 08:24:45.173007 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-598e8b5a-0717-4d84-9085-ec612d1bd5da None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 08:24:45.173007 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:45.173007 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:24:45.173007 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:24:45.173007 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:24:45.173007 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:24:45.173007 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:24:45.173007 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:24:45.173007 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:24:45.173007 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:24:45.173007 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:24:45.173007 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:24:45.173007 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 08:24:45.173007 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 08:24:45.173007 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 08:24:45.173007 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 08:24:45.174018 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 08:24:45.174018 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 08:24:45.174018 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 08:24:45.174018 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 08:24:45.174018 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:24:45.174018 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:24:45.174018 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:24:45.174018 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:24:45.174018 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:24:45.174018 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:24:45.174018 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:24:45.174018 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:24:45.174018 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:24:45.174018 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:24:45.174018 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:24:45.174018 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:24:45.174902 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:24:45.174902 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:24:45.174902 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:24:45.174902 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:24:45.174902 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 08:24:45.174902 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 08:24:45.174902 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 08:24:45.174902 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 08:24:45.174902 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 08:24:45.174902 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 08:24:45.174902 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 08:24:45.174902 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:24:45.174902 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 547/1087] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Aug 10 08:24:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:24:45.180387 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32a24cd-e9c2-4b54-920a-f2263f0cb48c None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:45.223049 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:45.279409 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32a24cd-e9c2-4b54-920a-f2263f0cb48c None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:45.299830 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587e9c31-daec-419c-8d6d-464f7002a84c None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:45.300244 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587e9c31-daec-419c-8d6d-464f7002a84c None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:45.300521 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587e9c31-daec-419c-8d6d-464f7002a84c None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:45.301886 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 548/1088] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:24:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 08:24:45.306803 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba65a79-da0f-4e22-aed5-ac4013017fd1 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:45.307091 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba65a79-da0f-4e22-aed5-ac4013017fd1 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:45.307367 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba65a79-da0f-4e22-aed5-ac4013017fd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:45.313754 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f56e2bf-b537-4bd4-9b2b-23ac50d041c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:45.314359 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f56e2bf-b537-4bd4-9b2b-23ac50d041c0 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:45.314662 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f56e2bf-b537-4bd4-9b2b-23ac50d041c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:45.314932 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f56e2bf-b537-4bd4-9b2b-23ac50d041c0 None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:45.315596 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f56e2bf-b537-4bd4-9b2b-23ac50d041c0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:45.315898 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f56e2bf-b537-4bd4-9b2b-23ac50d041c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:45.317304 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f56e2bf-b537-4bd4-9b2b-23ac50d041c0 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:45.325196 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-aba65a79-da0f-4e22-aed5-ac4013017fd1 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:24:45.325486 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 541/1089] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Aug 10 08:24:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:45.327602 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-aba65a79-da0f-4e22-aed5-ac4013017fd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:24:45.331153 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f56e2bf-b537-4bd4-9b2b-23ac50d041c0 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:45.352854 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:45.396834 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 549/1090] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:24:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 08:24:45.421086 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba65a79-da0f-4e22-aed5-ac4013017fd1 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:45.431752 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f56e2bf-b537-4bd4-9b2b-23ac50d041c0 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:45.434896 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:45.467407 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6379020e-aed0-4565-b877-92076b764a25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:45.468213 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6379020e-aed0-4565-b877-92076b764a25 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:45.468561 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6379020e-aed0-4565-b877-92076b764a25 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:45.468910 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6379020e-aed0-4565-b877-92076b764a25 None admin] PATH_INFO: `/v3/users/5e225fe0b5ba404e84e818b9e3bb40bf` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:45.470204 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6379020e-aed0-4565-b877-92076b764a25 None admin] RBAC: Authorizing `identity:get_user(user_id=5e225fe0b5ba404e84e818b9e3bb40bf)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:45.471636 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6379020e-aed0-4565-b877-92076b764a25 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:45.473933 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 542/1091] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Aug 10 08:24:45 2022] GET /identity/v3/users/5e225fe0b5ba404e84e818b9e3bb40bf => generated 285 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:24:45.479244 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6379020e-aed0-4565-b877-92076b764a25 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:45.522391 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba65a79-da0f-4e22-aed5-ac4013017fd1 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:45.556406 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c2403a-2413-460a-869c-4b555cd5d35c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:45.557056 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c2403a-2413-460a-869c-4b555cd5d35c None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:45.557332 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c2403a-2413-460a-869c-4b555cd5d35c None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:45.557597 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c2403a-2413-460a-869c-4b555cd5d35c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:45.568580 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c2403a-2413-460a-869c-4b555cd5d35c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 08:24:45.573142 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-49c2403a-2413-460a-869c-4b555cd5d35c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 08:24:45.573142 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:45.573142 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:24:45.573142 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:24:45.573142 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:24:45.573142 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:24:45.573142 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:24:45.573142 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:24:45.573142 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:24:45.573142 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:24:45.573142 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:24:45.573142 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:24:45.573142 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 08:24:45.573142 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 08:24:45.573142 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 08:24:45.573142 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 08:24:45.574201 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 08:24:45.574201 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 08:24:45.574201 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 08:24:45.574201 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 08:24:45.574201 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:24:45.574201 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:24:45.574201 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:24:45.574201 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:24:45.574201 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:24:45.574201 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:24:45.574201 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:24:45.574201 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:24:45.574201 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:24:45.574201 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:24:45.574201 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:24:45.574201 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:24:45.575067 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:24:45.575067 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:24:45.575067 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:24:45.575067 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:24:45.575067 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 08:24:45.575067 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 08:24:45.575067 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 08:24:45.575067 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 08:24:45.575067 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 08:24:45.575067 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 08:24:45.575067 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 08:24:45.575067 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:24:45.575067 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 550/1092] 10.222.0.57 () {56 vars in 1120 bytes} [Wed Aug 10 08:24:45 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 08:24:45.580702 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c2403a-2413-460a-869c-4b555cd5d35c None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:45.583006 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6379020e-aed0-4565-b877-92076b764a25 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:45.604134 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:45.616738 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a71ee6-b16e-4468-b48a-3162b4a1ddca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:45.617379 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a71ee6-b16e-4468-b48a-3162b4a1ddca None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:45.617661 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a71ee6-b16e-4468-b48a-3162b4a1ddca None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:45.617932 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a71ee6-b16e-4468-b48a-3162b4a1ddca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:45.658979 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:45.684704 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c2403a-2413-460a-869c-4b555cd5d35c None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:45.687611 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a71ee6-b16e-4468-b48a-3162b4a1ddca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:45.689267 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a71ee6-b16e-4468-b48a-3162b4a1ddca None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:45.718362 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ef76fb-1479-40ad-b1dc-9b64063057bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:45.718959 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ef76fb-1479-40ad-b1dc-9b64063057bd None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:45.719875 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ef76fb-1479-40ad-b1dc-9b64063057bd None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:45.720150 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ef76fb-1479-40ad-b1dc-9b64063057bd None admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:45.720809 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ef76fb-1479-40ad-b1dc-9b64063057bd None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:45.721145 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ef76fb-1479-40ad-b1dc-9b64063057bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:45.722502 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ef76fb-1479-40ad-b1dc-9b64063057bd None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:45.730637 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 551/1093] 10.222.0.57 () {56 vars in 1115 bytes} [Wed Aug 10 08:24:45 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:24:45.736623 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ef76fb-1479-40ad-b1dc-9b64063057bd None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:45.789936 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 543/1094] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Aug 10 08:24:45 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 10 08:24:45.795923 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a71ee6-b16e-4468-b48a-3162b4a1ddca None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:45.816134 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:45.833927 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ef76fb-1479-40ad-b1dc-9b64063057bd None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:45.862080 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:45.867987 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20555f5-cc65-427a-b655-4f6e1bf3c469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:45.868601 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20555f5-cc65-427a-b655-4f6e1bf3c469 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:45.869184 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20555f5-cc65-427a-b655-4f6e1bf3c469 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:45.869464 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20555f5-cc65-427a-b655-4f6e1bf3c469 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:45.879550 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20555f5-cc65-427a-b655-4f6e1bf3c469 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:45.881030 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20555f5-cc65-427a-b655-4f6e1bf3c469 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:45.892044 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a71ee6-b16e-4468-b48a-3162b4a1ddca None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:45.896659 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-c20555f5-cc65-427a-b655-4f6e1bf3c469 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:24:45.896659 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:45.896659 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:24:45.896659 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:24:45.896659 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:24:45.896659 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:24:45.896659 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:24:45.896659 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:24:45.896659 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:24:45.896659 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:24:45.896659 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:24:45.896659 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:24:45.896659 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 08:24:45.896659 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 08:24:45.896659 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 08:24:45.896659 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 08:24:45.897811 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:24:45.897811 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:24:45.897811 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:24:45.897811 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:24:45.897811 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:24:45.897811 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:24:45.897811 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:24:45.897811 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:24:45.897811 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:24:45.897811 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:24:45.897811 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:24:45.897811 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:24:45.897811 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:24:45.897811 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:24:45.897811 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:24:45.897811 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:24:45.897811 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 08:24:45.898922 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 08:24:45.898922 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:24:45.898922 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:24:45.902669 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 552/1095] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:24:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 08:24:45.903442 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20555f5-cc65-427a-b655-4f6e1bf3c469 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:45.927200 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d7371f-30c0-46d6-aae0-d236192a96db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:45.927818 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d7371f-30c0-46d6-aae0-d236192a96db None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:45.928091 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d7371f-30c0-46d6-aae0-d236192a96db None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:45.928356 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d7371f-30c0-46d6-aae0-d236192a96db None admin] PATH_INFO: `/v3/projects/e41afb9d169145b0abf42179e2a02bdf` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:45.929477 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d7371f-30c0-46d6-aae0-d236192a96db None admin] RBAC: Authorizing `identity:get_project(project_id=e41afb9d169145b0abf42179e2a02bdf)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:45.930891 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d7371f-30c0-46d6-aae0-d236192a96db None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:45.937584 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 544/1096] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Aug 10 08:24:45 2022] GET /identity/v3/projects/e41afb9d169145b0abf42179e2a02bdf => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:24:45.939374 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d7371f-30c0-46d6-aae0-d236192a96db None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:46.001637 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20555f5-cc65-427a-b655-4f6e1bf3c469 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:46.034655 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad64c089-10fb-4c2a-8c67-752423052c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:46.035254 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad64c089-10fb-4c2a-8c67-752423052c81 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:46.035532 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad64c089-10fb-4c2a-8c67-752423052c81 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:46.035801 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad64c089-10fb-4c2a-8c67-752423052c81 None admin] PATH_INFO: `/v3/domains` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:46.036436 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad64c089-10fb-4c2a-8c67-752423052c81 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:46.036530 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d7371f-30c0-46d6-aae0-d236192a96db None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:46.036734 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad64c089-10fb-4c2a-8c67-752423052c81 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:46.038113 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad64c089-10fb-4c2a-8c67-752423052c81 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:46.039528 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:46.050515 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 553/1097] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Aug 10 08:24:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:24:46.056107 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad64c089-10fb-4c2a-8c67-752423052c81 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:46.072276 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf18fe3-1802-486f-b2be-f1ce674e0c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:46.072873 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf18fe3-1802-486f-b2be-f1ce674e0c49 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:46.073224 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf18fe3-1802-486f-b2be-f1ce674e0c49 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:46.073485 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf18fe3-1802-486f-b2be-f1ce674e0c49 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:46.074195 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf18fe3-1802-486f-b2be-f1ce674e0c49 None admin] RBAC: Adding query filter params (role.id=ec3751692d9c4ecf81b4a71c936c46be, scope.project.id=e41afb9d169145b0abf42179e2a02bdf, user.id=5e225fe0b5ba404e84e818b9e3bb40bf) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:46.074496 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf18fe3-1802-486f-b2be-f1ce674e0c49 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:46.075893 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf18fe3-1802-486f-b2be-f1ce674e0c49 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:46.081928 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 545/1098] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Aug 10 08:24:45 2022] GET /identity/v3/role_assignments?user.id=5e225fe0b5ba404e84e818b9e3bb40bf&scope.project.id=e41afb9d169145b0abf42179e2a02bdf&role.id=ec3751692d9c4ecf81b4a71c936c46be => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:24:46.160326 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad64c089-10fb-4c2a-8c67-752423052c81 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:46.196860 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c9d824-0935-4079-acac-2ce045460e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:46.197438 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c9d824-0935-4079-acac-2ce045460e5e None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:46.197717 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c9d824-0935-4079-acac-2ce045460e5e None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:46.197986 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c9d824-0935-4079-acac-2ce045460e5e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:46.205846 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c9d824-0935-4079-acac-2ce045460e5e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:46.207308 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c9d824-0935-4079-acac-2ce045460e5e None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:46.220069 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-46c9d824-0935-4079-acac-2ce045460e5e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 08:24:46.220069 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:46.220069 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:24:46.220069 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:24:46.220069 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:24:46.220069 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:24:46.220069 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:24:46.220069 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:24:46.220069 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:24:46.220069 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:24:46.220069 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:24:46.220069 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:24:46.220069 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 08:24:46.220069 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 08:24:46.220069 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 08:24:46.220069 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 08:24:46.221202 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:24:46.221202 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:24:46.221202 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 08:24:46.221202 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 08:24:46.221202 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 08:24:46.221202 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 08:24:46.221202 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:24:46.221202 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:24:46.221202 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:24:46.221202 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:24:46.221202 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:24:46.221202 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:24:46.221202 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:24:46.221202 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:24:46.221202 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:24:46.221202 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:24:46.221202 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:24:46.221202 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:24:46.222576 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:24:46.222576 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:24:46.222576 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 08:24:46.222576 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 08:24:46.222576 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 08:24:46.222576 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 08:24:46.222576 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 08:24:46.222576 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 08:24:46.222576 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 08:24:46.222576 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:24:46.222576 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 554/1099] 10.222.0.57 () {56 vars in 1093 bytes} [Wed Aug 10 08:24:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 08:24:46.227072 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf18fe3-1802-486f-b2be-f1ce674e0c49 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:46.229947 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:46.330505 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf18fe3-1802-486f-b2be-f1ce674e0c49 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:46.366316 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a559085-fe84-42b3-b4be-d5c310f4206f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:46.366920 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a559085-fe84-42b3-b4be-d5c310f4206f None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:46.367199 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a559085-fe84-42b3-b4be-d5c310f4206f None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:46.367471 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a559085-fe84-42b3-b4be-d5c310f4206f None admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:46.368208 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a559085-fe84-42b3-b4be-d5c310f4206f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:46.368513 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a559085-fe84-42b3-b4be-d5c310f4206f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:46.370221 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a559085-fe84-42b3-b4be-d5c310f4206f None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:46.387540 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 546/1100] 10.222.0.57 () {56 vars in 1133 bytes} [Wed Aug 10 08:24:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:46.393596 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c9d824-0935-4079-acac-2ce045460e5e None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:46.448685 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:46.492625 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c9d824-0935-4079-acac-2ce045460e5e None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:46.526037 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b453ff74-ed86-44c5-87de-02f4fcb31f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:46.526638 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b453ff74-ed86-44c5-87de-02f4fcb31f4b None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:46.526917 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b453ff74-ed86-44c5-87de-02f4fcb31f4b None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:46.527186 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b453ff74-ed86-44c5-87de-02f4fcb31f4b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:46.593106 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b453ff74-ed86-44c5-87de-02f4fcb31f4b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:46.595310 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b453ff74-ed86-44c5-87de-02f4fcb31f4b None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:46.648445 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:46.692817 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 555/1101] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Aug 10 08:24:46 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 08:24:46.698577 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a559085-fe84-42b3-b4be-d5c310f4206f None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:46.756343 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:46.796545 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a559085-fe84-42b3-b4be-d5c310f4206f None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:46.831274 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a786d959-7a66-4019-bced-1d2642dfa484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:46.831894 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a786d959-7a66-4019-bced-1d2642dfa484 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:46.832169 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a786d959-7a66-4019-bced-1d2642dfa484 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:46.832434 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a786d959-7a66-4019-bced-1d2642dfa484 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:46.842719 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a786d959-7a66-4019-bced-1d2642dfa484 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:46.844182 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a786d959-7a66-4019-bced-1d2642dfa484 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:46.858707 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-a786d959-7a66-4019-bced-1d2642dfa484 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:24:46.858707 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:46.858707 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:24:46.858707 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:24:46.858707 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:24:46.858707 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:24:46.858707 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:24:46.858707 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:24:46.858707 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:24:46.858707 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:24:46.858707 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:24:46.858707 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:24:46.858707 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 08:24:46.858707 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 08:24:46.858707 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 08:24:46.858707 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 08:24:46.859814 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:24:46.859814 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:24:46.859814 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:24:46.859814 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:24:46.859814 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:24:46.859814 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:24:46.859814 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:24:46.859814 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:24:46.859814 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:24:46.859814 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:24:46.859814 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:24:46.859814 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:24:46.859814 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:24:46.859814 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:24:46.859814 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:24:46.859814 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:24:46.859814 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 08:24:46.860769 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 08:24:46.860769 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:24:46.860769 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:24:46.860769 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 547/1102] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:24:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:24:46.865669 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b453ff74-ed86-44c5-87de-02f4fcb31f4b None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:46.965764 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b453ff74-ed86-44c5-87de-02f4fcb31f4b None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:47.003902 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f010840-67e9-4d29-abe7-41236cbe421c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:47.004535 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f010840-67e9-4d29-abe7-41236cbe421c None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:47.004763 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f010840-67e9-4d29-abe7-41236cbe421c None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:47.005062 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f010840-67e9-4d29-abe7-41236cbe421c None admin] PATH_INFO: `/v3/domains` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:47.005698 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f010840-67e9-4d29-abe7-41236cbe421c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:47.005996 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f010840-67e9-4d29-abe7-41236cbe421c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:47.007412 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f010840-67e9-4d29-abe7-41236cbe421c None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:47.020199 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 556/1103] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Aug 10 08:24:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:47.025897 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a786d959-7a66-4019-bced-1d2642dfa484 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:47.127144 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a786d959-7a66-4019-bced-1d2642dfa484 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:47.139773 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:47.160117 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b40634-871f-4c5c-b086-6855702acbf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:47.160741 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b40634-871f-4c5c-b086-6855702acbf2 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:47.161053 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b40634-871f-4c5c-b086-6855702acbf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:47.161324 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b40634-871f-4c5c-b086-6855702acbf2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:47.167701 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b40634-871f-4c5c-b086-6855702acbf2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:47.169158 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b40634-871f-4c5c-b086-6855702acbf2 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:47.181261 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-96b40634-871f-4c5c-b086-6855702acbf2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 08:24:47.181261 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:24:47.181261 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:24:47.181261 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:24:47.181261 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:24:47.181261 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:24:47.181261 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:24:47.181261 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:24:47.181261 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:24:47.181261 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:24:47.181261 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:24:47.181261 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:24:47.181261 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 08:24:47.181261 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 08:24:47.181261 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 08:24:47.182214 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 08:24:47.182214 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:24:47.182214 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:24:47.182214 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:24:47.182214 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:24:47.182214 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:24:47.182214 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:24:47.182214 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:24:47.182214 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:24:47.182214 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:24:47.182214 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:24:47.182214 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:24:47.182214 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:24:47.182214 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:24:47.182214 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:24:47.182214 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:24:47.182214 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:24:47.183123 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 08:24:47.183123 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 08:24:47.183123 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 08:24:47.183123 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 08:24:47.183123 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 08:24:47.183123 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 08:24:47.183123 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 08:24:47.183123 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:24:47.183123 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 548/1104] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Aug 10 08:24:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:24:47.188731 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f010840-67e9-4d29-abe7-41236cbe421c None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:47.204569 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:47.289731 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f010840-67e9-4d29-abe7-41236cbe421c None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:47.324130 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac70bfc5-13f4-4cac-9912-d0764044b9c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:47.324731 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac70bfc5-13f4-4cac-9912-d0764044b9c3 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:47.325036 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac70bfc5-13f4-4cac-9912-d0764044b9c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:47.325308 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac70bfc5-13f4-4cac-9912-d0764044b9c3 None admin] PATH_INFO: `/v3/projects` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:47.325999 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac70bfc5-13f4-4cac-9912-d0764044b9c3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:47.326302 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac70bfc5-13f4-4cac-9912-d0764044b9c3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:47.327951 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac70bfc5-13f4-4cac-9912-d0764044b9c3 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:47.343402 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 557/1105] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Aug 10 08:24:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:47.350210 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b40634-871f-4c5c-b086-6855702acbf2 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:47.450365 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b40634-871f-4c5c-b086-6855702acbf2 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:47.488890 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1262b94-ba15-4c62-8219-51b710baf17d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:47.489511 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1262b94-ba15-4c62-8219-51b710baf17d None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:47.489786 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1262b94-ba15-4c62-8219-51b710baf17d None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:47.490054 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1262b94-ba15-4c62-8219-51b710baf17d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:47.490772 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1262b94-ba15-4c62-8219-51b710baf17d None admin] RBAC: Adding query filter params (role.id=c30407dbef394307b4d8d3dc9ca4bcaa, scope.project.id=b49e381419e940278bcb6111899589b5, user.id=a943008193914f76a663d489bd73e873) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:47.491074 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1262b94-ba15-4c62-8219-51b710baf17d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:47.492482 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1262b94-ba15-4c62-8219-51b710baf17d None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:47.498302 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 549/1106] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Aug 10 08:24:47 2022] GET /identity/v3/role_assignments?user.id=a943008193914f76a663d489bd73e873&scope.project.id=b49e381419e940278bcb6111899589b5&role.id=c30407dbef394307b4d8d3dc9ca4bcaa => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:48.143155 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d1ef9e-2f25-4c56-bad9-0c3a5ac18b48 None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:48.143678 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d1ef9e-2f25-4c56-bad9-0c3a5ac18b48 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:48.143678 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d1ef9e-2f25-4c56-bad9-0c3a5ac18b48 None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:48.145059 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 558/1107] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:24:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:24:48.150487 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd74b61-995e-4ad3-a0cb-ccc5a028c776 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:48.150966 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd74b61-995e-4ad3-a0cb-ccc5a028c776 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:48.151046 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd74b61-995e-4ad3-a0cb-ccc5a028c776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:48.170275 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-5dd74b61-995e-4ad3-a0cb-ccc5a028c776 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:24:48.171462 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-5dd74b61-995e-4ad3-a0cb-ccc5a028c776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:24:48.201531 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:48.240585 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 550/1108] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:24:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 08:24:48.264746 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d1ef9e-2f25-4c56-bad9-0c3a5ac18b48 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:48.342507 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:48.368141 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d1ef9e-2f25-4c56-bad9-0c3a5ac18b48 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:48.401274 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e3e8a8-8d3b-4d41-bec5-0857d9b91d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:48.401826 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e3e8a8-8d3b-4d41-bec5-0857d9b91d31 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:48.402098 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e3e8a8-8d3b-4d41-bec5-0857d9b91d31 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:48.402363 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e3e8a8-8d3b-4d41-bec5-0857d9b91d31 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:48.403463 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e3e8a8-8d3b-4d41-bec5-0857d9b91d31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:48.404875 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e3e8a8-8d3b-4d41-bec5-0857d9b91d31 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:48.407574 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 559/1109] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:24:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:48.412878 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd74b61-995e-4ad3-a0cb-ccc5a028c776 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:48.508036 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd74b61-995e-4ad3-a0cb-ccc5a028c776 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:48.540374 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7183bbf6-987b-4506-8510-c3e642594151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:48.540989 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7183bbf6-987b-4506-8510-c3e642594151 None admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:48.541268 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7183bbf6-987b-4506-8510-c3e642594151 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:48.541537 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7183bbf6-987b-4506-8510-c3e642594151 None admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:48.542166 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7183bbf6-987b-4506-8510-c3e642594151 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:48.543570 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7183bbf6-987b-4506-8510-c3e642594151 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:48.633046 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 551/1110] 10.222.0.57 () {60 vars in 1132 bytes} [Wed Aug 10 08:24:48 2022] POST /identity/v3/users => generated 286 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:24:50.670625 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b637306-af19-46aa-b2b8-d3fc8e622c1a None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:50.671143 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b637306-af19-46aa-b2b8-d3fc8e622c1a None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:50.671232 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b637306-af19-46aa-b2b8-d3fc8e622c1a None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:50.673215 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 560/1111] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:24:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:24:50.678373 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d918c5-29d7-4df7-90f3-0ff1b7ff5944 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:50.678776 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d918c5-29d7-4df7-90f3-0ff1b7ff5944 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:50.678914 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d918c5-29d7-4df7-90f3-0ff1b7ff5944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:50.696519 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-70d918c5-29d7-4df7-90f3-0ff1b7ff5944 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:24:50.697725 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-70d918c5-29d7-4df7-90f3-0ff1b7ff5944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:24:50.713720 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:50.762475 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 552/1112] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:24:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 08:24:50.785780 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b637306-af19-46aa-b2b8-d3fc8e622c1a None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:50.883136 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b637306-af19-46aa-b2b8-d3fc8e622c1a None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:50.915953 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d962a73-b4ff-4a00-a569-c25c3569aa68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:50.916568 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d962a73-b4ff-4a00-a569-c25c3569aa68 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:50.916846 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d962a73-b4ff-4a00-a569-c25c3569aa68 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:50.917145 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d962a73-b4ff-4a00-a569-c25c3569aa68 None admin] PATH_INFO: `/v3/roles/ccfc8d3fdf2d458299df29001278930f` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:50.918513 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d962a73-b4ff-4a00-a569-c25c3569aa68 None admin] RBAC: Authorizing `identity:get_role(role_id=ccfc8d3fdf2d458299df29001278930f)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:50.919868 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d962a73-b4ff-4a00-a569-c25c3569aa68 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:50.921265 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 561/1113] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Aug 10 08:24:50 2022] GET /identity/v3/roles/ccfc8d3fdf2d458299df29001278930f => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:50.927079 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d918c5-29d7-4df7-90f3-0ff1b7ff5944 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:51.022722 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d918c5-29d7-4df7-90f3-0ff1b7ff5944 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:51.055066 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ea80aa-39da-46ad-a7a1-dbc7f9832e6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:51.055679 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ea80aa-39da-46ad-a7a1-dbc7f9832e6a None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:51.055955 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ea80aa-39da-46ad-a7a1-dbc7f9832e6a None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:51.056223 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ea80aa-39da-46ad-a7a1-dbc7f9832e6a None admin] PATH_INFO: `/v3/users/ccb3df7fca1447dbb4599f8be0011e00` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:51.068797 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ea80aa-39da-46ad-a7a1-dbc7f9832e6a None admin] RBAC: Authorizing `identity:get_user(user_id=ccb3df7fca1447dbb4599f8be0011e00)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:51.070256 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ea80aa-39da-46ad-a7a1-dbc7f9832e6a None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:51.071820 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 553/1114] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Aug 10 08:24:50 2022] GET /identity/v3/users/ccb3df7fca1447dbb4599f8be0011e00 => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:51.077616 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d962a73-b4ff-4a00-a569-c25c3569aa68 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:51.090215 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:51.176838 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d962a73-b4ff-4a00-a569-c25c3569aa68 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:51.209461 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903ce573-9890-4fec-86b9-3ccc070b1ccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:51.210044 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903ce573-9890-4fec-86b9-3ccc070b1ccd None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:51.210317 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903ce573-9890-4fec-86b9-3ccc070b1ccd None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:51.210584 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903ce573-9890-4fec-86b9-3ccc070b1ccd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:51.275215 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903ce573-9890-4fec-86b9-3ccc070b1ccd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:51.276642 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903ce573-9890-4fec-86b9-3ccc070b1ccd None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:51.287662 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:51.371293 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 562/1115] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Aug 10 08:24:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 08:24:51.378246 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ea80aa-39da-46ad-a7a1-dbc7f9832e6a None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:51.390216 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:51.477524 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ea80aa-39da-46ad-a7a1-dbc7f9832e6a None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:51.511190 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22f1114-6b2a-4f85-bace-4209d4c1be0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:51.511797 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22f1114-6b2a-4f85-bace-4209d4c1be0f None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:51.512059 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22f1114-6b2a-4f85-bace-4209d4c1be0f None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:51.512325 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22f1114-6b2a-4f85-bace-4209d4c1be0f None admin] PATH_INFO: `/v3/projects/e41afb9d169145b0abf42179e2a02bdf` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:51.513442 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22f1114-6b2a-4f85-bace-4209d4c1be0f None admin] RBAC: Authorizing `identity:get_project(project_id=e41afb9d169145b0abf42179e2a02bdf)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:51.514850 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22f1114-6b2a-4f85-bace-4209d4c1be0f None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:51.517189 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 554/1116] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Aug 10 08:24:51 2022] GET /identity/v3/projects/e41afb9d169145b0abf42179e2a02bdf => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:51.523339 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903ce573-9890-4fec-86b9-3ccc070b1ccd None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:51.617782 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903ce573-9890-4fec-86b9-3ccc070b1ccd None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:51.629082 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:51.651034 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5010e273-7f22-4a8b-9463-0ff9579e5380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:51.651627 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5010e273-7f22-4a8b-9463-0ff9579e5380 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:51.651907 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5010e273-7f22-4a8b-9463-0ff9579e5380 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:51.652177 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5010e273-7f22-4a8b-9463-0ff9579e5380 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:51.652893 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5010e273-7f22-4a8b-9463-0ff9579e5380 None admin] RBAC: Adding query filter params (role.id=ccfc8d3fdf2d458299df29001278930f, scope.project.id=e41afb9d169145b0abf42179e2a02bdf, user.id=ccb3df7fca1447dbb4599f8be0011e00) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:51.653230 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5010e273-7f22-4a8b-9463-0ff9579e5380 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:51.654642 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5010e273-7f22-4a8b-9463-0ff9579e5380 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:51.660098 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 563/1117] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Aug 10 08:24:51 2022] GET /identity/v3/role_assignments?user.id=ccb3df7fca1447dbb4599f8be0011e00&scope.project.id=e41afb9d169145b0abf42179e2a02bdf&role.id=ccfc8d3fdf2d458299df29001278930f => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:53.636401 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dbc82f-a693-426d-9199-8cefbfdac4d5 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:53.637011 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dbc82f-a693-426d-9199-8cefbfdac4d5 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:53.637011 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dbc82f-a693-426d-9199-8cefbfdac4d5 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:53.638291 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 555/1118] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:24:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:24:53.643787 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2d3c30-dff5-449d-b47d-98048070a19e None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:53.644228 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2d3c30-dff5-449d-b47d-98048070a19e None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:53.644329 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2d3c30-dff5-449d-b47d-98048070a19e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:53.662463 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-2a2d3c30-dff5-449d-b47d-98048070a19e None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:24:53.663643 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-2a2d3c30-dff5-449d-b47d-98048070a19e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:24:53.727403 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 564/1119] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:24:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 08:24:53.749855 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dbc82f-a693-426d-9199-8cefbfdac4d5 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:53.825762 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:53.847564 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dbc82f-a693-426d-9199-8cefbfdac4d5 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:53.881198 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3553eaf-f2a9-4b15-8d2f-3d560af78eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:53.881806 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3553eaf-f2a9-4b15-8d2f-3d560af78eeb None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:53.882079 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3553eaf-f2a9-4b15-8d2f-3d560af78eeb None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:53.882343 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3553eaf-f2a9-4b15-8d2f-3d560af78eeb None admin] PATH_INFO: `/v3/roles/ccfc8d3fdf2d458299df29001278930f` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:53.883692 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3553eaf-f2a9-4b15-8d2f-3d560af78eeb None admin] RBAC: Authorizing `identity:get_role(role_id=ccfc8d3fdf2d458299df29001278930f)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:53.885070 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3553eaf-f2a9-4b15-8d2f-3d560af78eeb None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:53.886374 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 556/1120] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Aug 10 08:24:53 2022] GET /identity/v3/roles/ccfc8d3fdf2d458299df29001278930f => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:53.892605 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2d3c30-dff5-449d-b47d-98048070a19e None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:53.987921 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2d3c30-dff5-449d-b47d-98048070a19e None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:53.990102 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:54.020689 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8419f6e8-1588-4d67-b372-6c2521e1aecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:54.021533 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8419f6e8-1588-4d67-b372-6c2521e1aecc None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:54.022128 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8419f6e8-1588-4d67-b372-6c2521e1aecc None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:54.022410 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8419f6e8-1588-4d67-b372-6c2521e1aecc None admin] PATH_INFO: `/v3/users/ccb3df7fca1447dbb4599f8be0011e00` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:54.023612 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8419f6e8-1588-4d67-b372-6c2521e1aecc None admin] RBAC: Authorizing `identity:get_user(user_id=ccb3df7fca1447dbb4599f8be0011e00)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:54.025084 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8419f6e8-1588-4d67-b372-6c2521e1aecc None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:54.026672 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 565/1121] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Aug 10 08:24:53 2022] GET /identity/v3/users/ccb3df7fca1447dbb4599f8be0011e00 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:54.033295 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3553eaf-f2a9-4b15-8d2f-3d560af78eeb None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:54.128786 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3553eaf-f2a9-4b15-8d2f-3d560af78eeb None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:54.162408 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832ff1cd-db15-4e78-9405-bb194519796a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:54.163019 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832ff1cd-db15-4e78-9405-bb194519796a None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:54.163299 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832ff1cd-db15-4e78-9405-bb194519796a None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:54.163574 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832ff1cd-db15-4e78-9405-bb194519796a None admin] PATH_INFO: `/v3/projects/e41afb9d169145b0abf42179e2a02bdf` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:54.164675 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832ff1cd-db15-4e78-9405-bb194519796a None admin] RBAC: Authorizing `identity:get_project(project_id=e41afb9d169145b0abf42179e2a02bdf)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:54.166129 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832ff1cd-db15-4e78-9405-bb194519796a None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:54.167762 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 557/1122] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Aug 10 08:24:54 2022] GET /identity/v3/projects/e41afb9d169145b0abf42179e2a02bdf => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:54.173813 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8419f6e8-1588-4d67-b372-6c2521e1aecc None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:54.269462 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8419f6e8-1588-4d67-b372-6c2521e1aecc None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:54.302329 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ca9503-382d-4427-8951-73a820ff265d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:54.302961 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ca9503-382d-4427-8951-73a820ff265d None admin] REQUEST_METHOD: `PUT` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:54.303237 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ca9503-382d-4427-8951-73a820ff265d None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:54.303505 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ca9503-382d-4427-8951-73a820ff265d None admin] PATH_INFO: `/v3/projects/e41afb9d169145b0abf42179e2a02bdf/users/ccb3df7fca1447dbb4599f8be0011e00/roles/ccfc8d3fdf2d458299df29001278930f` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:54.305724 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ca9503-382d-4427-8951-73a820ff265d None admin] RBAC: Authorizing `identity:create_grant(project_id=e41afb9d169145b0abf42179e2a02bdf, user_id=ccb3df7fca1447dbb4599f8be0011e00, role_id=ccfc8d3fdf2d458299df29001278930f)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:54.307131 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ca9503-382d-4427-8951-73a820ff265d None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:54.404727 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 566/1123] 10.222.0.57 () {58 vars in 1439 bytes} [Wed Aug 10 08:24:54 2022] PUT /identity/v3/projects/e41afb9d169145b0abf42179e2a02bdf/users/ccb3df7fca1447dbb4599f8be0011e00/roles/ccfc8d3fdf2d458299df29001278930f => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:24:56.391683 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf28057f-8cf6-4ba5-9f0a-2173fde2d680 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:56.392234 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf28057f-8cf6-4ba5-9f0a-2173fde2d680 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:56.392234 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf28057f-8cf6-4ba5-9f0a-2173fde2d680 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:56.393600 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 558/1124] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:24:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:24:56.398605 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0f9dcb-4f38-4c2e-8670-51c9a1ca04f7 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:56.399015 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0f9dcb-4f38-4c2e-8670-51c9a1ca04f7 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:56.399153 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0f9dcb-4f38-4c2e-8670-51c9a1ca04f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:56.417979 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-fe0f9dcb-4f38-4c2e-8670-51c9a1ca04f7 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:24:56.419152 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-fe0f9dcb-4f38-4c2e-8670-51c9a1ca04f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:24:56.444377 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:56.484639 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 567/1125] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:24:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 08:24:56.508721 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf28057f-8cf6-4ba5-9f0a-2173fde2d680 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:56.524618 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:56.606393 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf28057f-8cf6-4ba5-9f0a-2173fde2d680 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:56.639612 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0d2038-93c9-40fd-9b1a-c9a62d38ab13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:56.640221 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0d2038-93c9-40fd-9b1a-c9a62d38ab13 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:56.641393 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0d2038-93c9-40fd-9b1a-c9a62d38ab13 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:56.641520 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0d2038-93c9-40fd-9b1a-c9a62d38ab13 None admin] PATH_INFO: `/v3/roles/ccfc8d3fdf2d458299df29001278930f` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:56.642945 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0d2038-93c9-40fd-9b1a-c9a62d38ab13 None admin] RBAC: Authorizing `identity:get_role(role_id=ccfc8d3fdf2d458299df29001278930f)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:56.644246 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0d2038-93c9-40fd-9b1a-c9a62d38ab13 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:56.645569 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 559/1126] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Aug 10 08:24:56 2022] GET /identity/v3/roles/ccfc8d3fdf2d458299df29001278930f => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:56.651301 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0f9dcb-4f38-4c2e-8670-51c9a1ca04f7 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:56.748824 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0f9dcb-4f38-4c2e-8670-51c9a1ca04f7 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:56.782513 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d3e843-f4b0-443e-beba-944dd1b9b43c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:56.783134 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d3e843-f4b0-443e-beba-944dd1b9b43c None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:56.783410 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d3e843-f4b0-443e-beba-944dd1b9b43c None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:56.783679 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d3e843-f4b0-443e-beba-944dd1b9b43c None admin] PATH_INFO: `/v3/users/ccb3df7fca1447dbb4599f8be0011e00` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:56.784788 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d3e843-f4b0-443e-beba-944dd1b9b43c None admin] RBAC: Authorizing `identity:get_user(user_id=ccb3df7fca1447dbb4599f8be0011e00)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:56.786248 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d3e843-f4b0-443e-beba-944dd1b9b43c None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:56.787819 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 568/1127] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Aug 10 08:24:56 2022] GET /identity/v3/users/ccb3df7fca1447dbb4599f8be0011e00 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:56.793655 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0d2038-93c9-40fd-9b1a-c9a62d38ab13 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:56.872725 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:56.890114 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0d2038-93c9-40fd-9b1a-c9a62d38ab13 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:56.923755 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508a18e8-af8d-4c97-acd7-b6e4dc379869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:56.924346 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508a18e8-af8d-4c97-acd7-b6e4dc379869 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:56.924621 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508a18e8-af8d-4c97-acd7-b6e4dc379869 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:56.924882 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508a18e8-af8d-4c97-acd7-b6e4dc379869 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:56.988831 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508a18e8-af8d-4c97-acd7-b6e4dc379869 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:56.990289 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508a18e8-af8d-4c97-acd7-b6e4dc379869 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:57.069023 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:57.086705 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 560/1128] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Aug 10 08:24:56 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 08:24:57.092708 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d3e843-f4b0-443e-beba-944dd1b9b43c None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:57.122782 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:57.191064 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d3e843-f4b0-443e-beba-944dd1b9b43c None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:57.223445 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb7b5ba-97c0-40f9-9164-6c7ec9e33fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:57.224058 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb7b5ba-97c0-40f9-9164-6c7ec9e33fde None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:57.224338 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb7b5ba-97c0-40f9-9164-6c7ec9e33fde None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:57.224608 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb7b5ba-97c0-40f9-9164-6c7ec9e33fde None admin] PATH_INFO: `/v3/projects/e41afb9d169145b0abf42179e2a02bdf` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:57.225749 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb7b5ba-97c0-40f9-9164-6c7ec9e33fde None admin] RBAC: Authorizing `identity:get_project(project_id=e41afb9d169145b0abf42179e2a02bdf)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:57.227169 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb7b5ba-97c0-40f9-9164-6c7ec9e33fde None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:57.228783 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 569/1129] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Aug 10 08:24:57 2022] GET /identity/v3/projects/e41afb9d169145b0abf42179e2a02bdf => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:57.235543 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508a18e8-af8d-4c97-acd7-b6e4dc379869 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:57.330204 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508a18e8-af8d-4c97-acd7-b6e4dc379869 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:57.363208 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32c356e-b8b2-4c62-a5e6-fb5c33955dc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:57.363790 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32c356e-b8b2-4c62-a5e6-fb5c33955dc8 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:57.364063 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32c356e-b8b2-4c62-a5e6-fb5c33955dc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:57.364330 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32c356e-b8b2-4c62-a5e6-fb5c33955dc8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:57.365067 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32c356e-b8b2-4c62-a5e6-fb5c33955dc8 None admin] RBAC: Adding query filter params (role.id=ccfc8d3fdf2d458299df29001278930f, scope.project.id=e41afb9d169145b0abf42179e2a02bdf, user.id=ccb3df7fca1447dbb4599f8be0011e00) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:24:57.365369 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32c356e-b8b2-4c62-a5e6-fb5c33955dc8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:57.366775 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32c356e-b8b2-4c62-a5e6-fb5c33955dc8 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:57.372566 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 561/1130] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Aug 10 08:24:57 2022] GET /identity/v3/role_assignments?user.id=ccb3df7fca1447dbb4599f8be0011e00&scope.project.id=e41afb9d169145b0abf42179e2a02bdf&role.id=ccfc8d3fdf2d458299df29001278930f => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:59.401328 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49f81fd-ae74-4a0d-a4ce-449bc11b2f54 None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:59.401877 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49f81fd-ae74-4a0d-a4ce-449bc11b2f54 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:59.401877 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49f81fd-ae74-4a0d-a4ce-449bc11b2f54 None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:59.403189 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 570/1131] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:24:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:24:59.409129 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98b1070-ec34-456b-aa03-bb23dd10c5fb None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:59.409568 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98b1070-ec34-456b-aa03-bb23dd10c5fb None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:59.409680 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98b1070-ec34-456b-aa03-bb23dd10c5fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:59.427896 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-e98b1070-ec34-456b-aa03-bb23dd10c5fb None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:24:59.429105 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-e98b1070-ec34-456b-aa03-bb23dd10c5fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:24:59.470216 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:59.493766 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 562/1132] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:24:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 08:24:59.517962 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49f81fd-ae74-4a0d-a4ce-449bc11b2f54 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:59.612014 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:24:59.616468 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49f81fd-ae74-4a0d-a4ce-449bc11b2f54 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:59.651108 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa643f73-d92d-4bdd-80cf-5e4c6f2029e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:59.651770 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa643f73-d92d-4bdd-80cf-5e4c6f2029e6 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:59.651994 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa643f73-d92d-4bdd-80cf-5e4c6f2029e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:59.652258 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa643f73-d92d-4bdd-80cf-5e4c6f2029e6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:59.653390 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa643f73-d92d-4bdd-80cf-5e4c6f2029e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:59.654799 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa643f73-d92d-4bdd-80cf-5e4c6f2029e6 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:59.656476 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 571/1133] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:24:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:24:59.662368 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98b1070-ec34-456b-aa03-bb23dd10c5fb None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:24:59.757174 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98b1070-ec34-456b-aa03-bb23dd10c5fb None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:24:59.789905 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8965e6a5-7ef3-4957-83fc-8108859b712c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:24:59.790494 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8965e6a5-7ef3-4957-83fc-8108859b712c None admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:24:59.790772 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8965e6a5-7ef3-4957-83fc-8108859b712c None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:24:59.791041 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8965e6a5-7ef3-4957-83fc-8108859b712c None admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:24:59.791679 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8965e6a5-7ef3-4957-83fc-8108859b712c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:24:59.793120 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8965e6a5-7ef3-4957-83fc-8108859b712c None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:24:59.902383 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 563/1134] 10.222.0.57 () {60 vars in 1141 bytes} [Wed Aug 10 08:24:59 2022] POST /identity/v3/projects => generated 311 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:25:01.904702 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c93cd7-4b00-4145-b3bb-976264d0e00f None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:25:01.905316 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c93cd7-4b00-4145-b3bb-976264d0e00f None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:25:01.905316 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c93cd7-4b00-4145-b3bb-976264d0e00f None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:25:01.906880 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 572/1135] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:25:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:25:01.912057 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f242b9d1-d0a0-42d6-90f0-dd7566d24f25 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:25:01.912472 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f242b9d1-d0a0-42d6-90f0-dd7566d24f25 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:25:01.912598 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f242b9d1-d0a0-42d6-90f0-dd7566d24f25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:25:01.930344 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-f242b9d1-d0a0-42d6-90f0-dd7566d24f25 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:25:01.931514 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-f242b9d1-d0a0-42d6-90f0-dd7566d24f25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:25:01.954320 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:25:01.996787 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 564/1136] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:25:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 08:25:02.022792 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c93cd7-4b00-4145-b3bb-976264d0e00f None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:25:02.120283 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c93cd7-4b00-4145-b3bb-976264d0e00f None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:25:02.153769 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105bef17-a566-49e7-82ea-1b2227c023bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:25:02.154383 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105bef17-a566-49e7-82ea-1b2227c023bd None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:25:02.154660 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105bef17-a566-49e7-82ea-1b2227c023bd None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:25:02.154975 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105bef17-a566-49e7-82ea-1b2227c023bd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:25:02.156034 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105bef17-a566-49e7-82ea-1b2227c023bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:25:02.157484 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105bef17-a566-49e7-82ea-1b2227c023bd None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:25:02.159045 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 573/1137] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:25:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:25:02.164714 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f242b9d1-d0a0-42d6-90f0-dd7566d24f25 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:25:02.262536 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f242b9d1-d0a0-42d6-90f0-dd7566d24f25 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:25:02.294981 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47060ac7-7863-4f11-af60-f0ed101bfabb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:25:02.295572 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47060ac7-7863-4f11-af60-f0ed101bfabb None admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:25:02.295846 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47060ac7-7863-4f11-af60-f0ed101bfabb None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:25:02.296112 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47060ac7-7863-4f11-af60-f0ed101bfabb None admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:25:02.297697 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47060ac7-7863-4f11-af60-f0ed101bfabb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:25:02.299110 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47060ac7-7863-4f11-af60-f0ed101bfabb None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:25:02.381292 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 565/1138] 10.222.0.57 () {60 vars in 1132 bytes} [Wed Aug 10 08:25:02 2022] POST /identity/v3/users => generated 286 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:25:04.425709 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66129ccf-33ef-4fcb-b522-3a82632e84d7 None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:25:04.426132 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66129ccf-33ef-4fcb-b522-3a82632e84d7 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:25:04.426252 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66129ccf-33ef-4fcb-b522-3a82632e84d7 None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:25:04.427617 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 574/1139] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:25:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:25:04.432725 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f6346f-89f8-4f47-b6ae-9e900c3474bb None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:25:04.433177 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f6346f-89f8-4f47-b6ae-9e900c3474bb None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:25:04.433287 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f6346f-89f8-4f47-b6ae-9e900c3474bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:25:04.451378 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-b0f6346f-89f8-4f47-b6ae-9e900c3474bb None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:25:04.453212 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-b0f6346f-89f8-4f47-b6ae-9e900c3474bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:25:04.462145 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:25:04.519322 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 566/1140] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:25:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 08:25:04.542270 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66129ccf-33ef-4fcb-b522-3a82632e84d7 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:25:04.575511 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:25:04.638802 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66129ccf-33ef-4fcb-b522-3a82632e84d7 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:25:04.672030 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af199096-0c09-460f-978a-6f577fd6d627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:25:04.672642 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af199096-0c09-460f-978a-6f577fd6d627 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:25:04.672946 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af199096-0c09-460f-978a-6f577fd6d627 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:25:04.673222 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af199096-0c09-460f-978a-6f577fd6d627 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:25:04.682968 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af199096-0c09-460f-978a-6f577fd6d627 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 08:25:04.688322 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-af199096-0c09-460f-978a-6f577fd6d627 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 08:25:04.688322 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:25:04.688322 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:25:04.688322 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:25:04.688322 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:25:04.688322 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:25:04.688322 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:25:04.688322 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:25:04.688322 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:25:04.688322 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:25:04.688322 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:25:04.688322 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:25:04.688322 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 08:25:04.688322 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 08:25:04.688322 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 08:25:04.688322 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 08:25:04.689445 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 08:25:04.689445 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 08:25:04.689445 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 08:25:04.689445 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 08:25:04.689445 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:25:04.689445 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:25:04.689445 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:25:04.689445 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:25:04.689445 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:25:04.689445 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:25:04.689445 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:25:04.689445 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:25:04.689445 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:25:04.689445 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:25:04.689445 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:25:04.689445 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:25:04.690376 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:25:04.690376 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:25:04.690376 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:25:04.690376 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:25:04.690376 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 08:25:04.690376 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 08:25:04.690376 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 08:25:04.690376 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 08:25:04.690376 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 08:25:04.690376 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 08:25:04.690376 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 08:25:04.690376 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:25:04.690376 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 575/1141] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Aug 10 08:25:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:25:04.695437 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f6346f-89f8-4f47-b6ae-9e900c3474bb None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:25:04.790851 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f6346f-89f8-4f47-b6ae-9e900c3474bb None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:25:04.823893 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3363a318-759b-4916-81e4-f21ecac068c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:25:04.824493 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3363a318-759b-4916-81e4-f21ecac068c4 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:25:04.824772 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3363a318-759b-4916-81e4-f21ecac068c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:25:04.825077 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3363a318-759b-4916-81e4-f21ecac068c4 None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:25:04.825737 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3363a318-759b-4916-81e4-f21ecac068c4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:25:04.826041 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3363a318-759b-4916-81e4-f21ecac068c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:25:04.828030 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3363a318-759b-4916-81e4-f21ecac068c4 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:25:04.835910 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 567/1142] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Aug 10 08:25:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:25:04.841620 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af199096-0c09-460f-978a-6f577fd6d627 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:25:04.931215 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:25:04.937948 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af199096-0c09-460f-978a-6f577fd6d627 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:25:04.970466 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85afeb0e-ee2b-4a26-99fa-9ab37fc62c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:25:04.971076 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85afeb0e-ee2b-4a26-99fa-9ab37fc62c07 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:25:04.971354 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85afeb0e-ee2b-4a26-99fa-9ab37fc62c07 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:25:04.971625 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85afeb0e-ee2b-4a26-99fa-9ab37fc62c07 None admin] PATH_INFO: `/v3/users/23c67fddcd10410a8fe50561653558ef` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:25:04.984300 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85afeb0e-ee2b-4a26-99fa-9ab37fc62c07 None admin] RBAC: Authorizing `identity:get_user(user_id=23c67fddcd10410a8fe50561653558ef)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:25:04.985888 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85afeb0e-ee2b-4a26-99fa-9ab37fc62c07 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:25:04.987479 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 576/1143] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Aug 10 08:25:04 2022] GET /identity/v3/users/23c67fddcd10410a8fe50561653558ef => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:25:04.993299 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3363a318-759b-4916-81e4-f21ecac068c4 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:25:05.003968 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:25:05.094636 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3363a318-759b-4916-81e4-f21ecac068c4 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:25:05.128068 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfb25b9-6ffb-4dd2-bb27-29210ecd2d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:25:05.128664 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfb25b9-6ffb-4dd2-bb27-29210ecd2d86 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:25:05.128962 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfb25b9-6ffb-4dd2-bb27-29210ecd2d86 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:25:05.129240 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfb25b9-6ffb-4dd2-bb27-29210ecd2d86 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:25:05.194589 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfb25b9-6ffb-4dd2-bb27-29210ecd2d86 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:25:05.195980 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfb25b9-6ffb-4dd2-bb27-29210ecd2d86 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:25:05.203844 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:25:05.292128 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 568/1144] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Aug 10 08:25:04 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 08:25:05.298685 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85afeb0e-ee2b-4a26-99fa-9ab37fc62c07 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:25:05.394504 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85afeb0e-ee2b-4a26-99fa-9ab37fc62c07 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:25:05.427935 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e8c1f3-6f60-40d9-8a5b-bc7d7724fbc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:25:05.428556 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e8c1f3-6f60-40d9-8a5b-bc7d7724fbc4 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:25:05.428833 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e8c1f3-6f60-40d9-8a5b-bc7d7724fbc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:25:05.429133 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e8c1f3-6f60-40d9-8a5b-bc7d7724fbc4 None admin] PATH_INFO: `/v3/projects/eb0de551a4aa4b6dad55975a1913c1b9` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:25:05.430231 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e8c1f3-6f60-40d9-8a5b-bc7d7724fbc4 None admin] RBAC: Authorizing `identity:get_project(project_id=eb0de551a4aa4b6dad55975a1913c1b9)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:25:05.431683 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e8c1f3-6f60-40d9-8a5b-bc7d7724fbc4 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:25:05.433344 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 577/1145] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Aug 10 08:25:05 2022] GET /identity/v3/projects/eb0de551a4aa4b6dad55975a1913c1b9 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:25:05.440166 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfb25b9-6ffb-4dd2-bb27-29210ecd2d86 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:25:05.537710 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfb25b9-6ffb-4dd2-bb27-29210ecd2d86 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:25:05.544760 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:25:05.571190 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc43d14-feed-432e-891a-d2f83b8a1f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:25:05.571786 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc43d14-feed-432e-891a-d2f83b8a1f30 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:25:05.572064 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc43d14-feed-432e-891a-d2f83b8a1f30 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:25:05.572334 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc43d14-feed-432e-891a-d2f83b8a1f30 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:25:05.573090 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc43d14-feed-432e-891a-d2f83b8a1f30 None admin] RBAC: Adding query filter params (role.id=ec3751692d9c4ecf81b4a71c936c46be, scope.project.id=eb0de551a4aa4b6dad55975a1913c1b9, user.id=23c67fddcd10410a8fe50561653558ef) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:25:05.573398 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc43d14-feed-432e-891a-d2f83b8a1f30 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:25:05.574824 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc43d14-feed-432e-891a-d2f83b8a1f30 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:25:05.580276 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 569/1146] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Aug 10 08:25:05 2022] GET /identity/v3/role_assignments?user.id=23c67fddcd10410a8fe50561653558ef&scope.project.id=eb0de551a4aa4b6dad55975a1913c1b9&role.id=ec3751692d9c4ecf81b4a71c936c46be => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:25:07.545541 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2160b56-32db-4f21-a435-736c5d9012f2 None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:25:07.546635 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2160b56-32db-4f21-a435-736c5d9012f2 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:25:07.546742 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2160b56-32db-4f21-a435-736c5d9012f2 None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:25:07.548019 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 578/1147] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:25:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:25:07.553209 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4859bd98-d355-4cfa-a328-d9bb914b988d None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:25:07.553611 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4859bd98-d355-4cfa-a328-d9bb914b988d None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:25:07.553755 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4859bd98-d355-4cfa-a328-d9bb914b988d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:25:07.572287 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-4859bd98-d355-4cfa-a328-d9bb914b988d None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:25:07.573491 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-4859bd98-d355-4cfa-a328-d9bb914b988d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:25:07.640036 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 570/1148] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:25:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 08:25:07.668679 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2160b56-32db-4f21-a435-736c5d9012f2 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:25:07.690504 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:25:07.766689 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2160b56-32db-4f21-a435-736c5d9012f2 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:25:07.799577 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22352b57-5bc2-429f-ac7b-049d1fee0333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:25:07.800180 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22352b57-5bc2-429f-ac7b-049d1fee0333 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:25:07.800453 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22352b57-5bc2-429f-ac7b-049d1fee0333 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:25:07.800719 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22352b57-5bc2-429f-ac7b-049d1fee0333 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:25:07.810288 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22352b57-5bc2-429f-ac7b-049d1fee0333 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 08:25:07.814867 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-22352b57-5bc2-429f-ac7b-049d1fee0333 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 08:25:07.814867 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:25:07.814867 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:25:07.814867 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:25:07.814867 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:25:07.814867 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:25:07.814867 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:25:07.814867 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:25:07.814867 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:25:07.814867 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:25:07.814867 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:25:07.814867 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:25:07.814867 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 08:25:07.814867 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 08:25:07.814867 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 08:25:07.814867 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 08:25:07.815968 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 08:25:07.815968 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 08:25:07.815968 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 08:25:07.815968 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 08:25:07.815968 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:25:07.815968 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:25:07.815968 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:25:07.815968 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:25:07.815968 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:25:07.815968 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:25:07.815968 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:25:07.815968 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:25:07.815968 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:25:07.815968 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:25:07.815968 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:25:07.815968 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:25:07.817158 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:25:07.817158 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:25:07.817158 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:25:07.817158 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:25:07.817158 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 08:25:07.817158 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 08:25:07.817158 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 08:25:07.817158 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 08:25:07.817158 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 08:25:07.817158 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 08:25:07.817158 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 08:25:07.817158 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:25:07.817158 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 579/1149] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Aug 10 08:25:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:25:07.821947 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4859bd98-d355-4cfa-a328-d9bb914b988d None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:25:07.914906 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:25:07.917760 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4859bd98-d355-4cfa-a328-d9bb914b988d None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:25:07.951187 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bd617a-176e-4b19-b25a-861a48ea72d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:25:07.951726 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bd617a-176e-4b19-b25a-861a48ea72d9 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:25:07.952005 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bd617a-176e-4b19-b25a-861a48ea72d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:25:07.952373 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bd617a-176e-4b19-b25a-861a48ea72d9 None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:25:07.953078 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bd617a-176e-4b19-b25a-861a48ea72d9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:25:07.953384 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bd617a-176e-4b19-b25a-861a48ea72d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:25:07.954745 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bd617a-176e-4b19-b25a-861a48ea72d9 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:25:07.962463 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 571/1150] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Aug 10 08:25:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:25:07.968753 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22352b57-5bc2-429f-ac7b-049d1fee0333 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:25:08.048092 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:25:08.066228 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22352b57-5bc2-429f-ac7b-049d1fee0333 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:25:08.099590 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7633a59-208a-45a0-8190-bd904af0b813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:25:08.100203 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7633a59-208a-45a0-8190-bd904af0b813 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:25:08.100483 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7633a59-208a-45a0-8190-bd904af0b813 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:25:08.100755 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7633a59-208a-45a0-8190-bd904af0b813 None admin] PATH_INFO: `/v3/users/23c67fddcd10410a8fe50561653558ef` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:25:08.101896 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7633a59-208a-45a0-8190-bd904af0b813 None admin] RBAC: Authorizing `identity:get_user(user_id=23c67fddcd10410a8fe50561653558ef)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:25:08.103320 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7633a59-208a-45a0-8190-bd904af0b813 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:25:08.104880 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 580/1151] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Aug 10 08:25:07 2022] GET /identity/v3/users/23c67fddcd10410a8fe50561653558ef => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:25:08.111015 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bd617a-176e-4b19-b25a-861a48ea72d9 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:25:08.208525 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bd617a-176e-4b19-b25a-861a48ea72d9 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:25:08.240737 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e85ad7f-f54f-4050-8f4b-5692fedabbf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:25:08.241373 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e85ad7f-f54f-4050-8f4b-5692fedabbf7 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:25:08.241653 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e85ad7f-f54f-4050-8f4b-5692fedabbf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:25:08.241923 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e85ad7f-f54f-4050-8f4b-5692fedabbf7 None admin] PATH_INFO: `/v3/projects/eb0de551a4aa4b6dad55975a1913c1b9` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:25:08.243017 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e85ad7f-f54f-4050-8f4b-5692fedabbf7 None admin] RBAC: Authorizing `identity:get_project(project_id=eb0de551a4aa4b6dad55975a1913c1b9)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:25:08.244690 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e85ad7f-f54f-4050-8f4b-5692fedabbf7 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:25:08.246301 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 572/1152] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Aug 10 08:25:08 2022] GET /identity/v3/projects/eb0de551a4aa4b6dad55975a1913c1b9 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:25:08.252100 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7633a59-208a-45a0-8190-bd904af0b813 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:25:08.348103 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7633a59-208a-45a0-8190-bd904af0b813 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:25:08.382330 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb72716-a96d-4f17-ad55-820723bb076e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:25:08.382972 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb72716-a96d-4f17-ad55-820723bb076e None admin] REQUEST_METHOD: `PUT` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:25:08.383253 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb72716-a96d-4f17-ad55-820723bb076e None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:25:08.383526 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb72716-a96d-4f17-ad55-820723bb076e None admin] PATH_INFO: `/v3/projects/eb0de551a4aa4b6dad55975a1913c1b9/users/23c67fddcd10410a8fe50561653558ef/roles/ec3751692d9c4ecf81b4a71c936c46be` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:25:08.385469 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb72716-a96d-4f17-ad55-820723bb076e None admin] RBAC: Authorizing `identity:create_grant(project_id=eb0de551a4aa4b6dad55975a1913c1b9, user_id=23c67fddcd10410a8fe50561653558ef, role_id=ec3751692d9c4ecf81b4a71c936c46be)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:25:08.386908 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb72716-a96d-4f17-ad55-820723bb076e None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:25:08.460658 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 581/1153] 10.222.0.57 () {58 vars in 1439 bytes} [Wed Aug 10 08:25:08 2022] PUT /identity/v3/projects/eb0de551a4aa4b6dad55975a1913c1b9/users/23c67fddcd10410a8fe50561653558ef/roles/ec3751692d9c4ecf81b4a71c936c46be => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:25:10.526426 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abc45a5-4ffb-442e-aa4c-b46642394b5a None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:25:10.526963 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abc45a5-4ffb-442e-aa4c-b46642394b5a None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:25:10.526963 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abc45a5-4ffb-442e-aa4c-b46642394b5a None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:25:10.528290 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 573/1154] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:25:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:25:10.534151 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2c6a48-2a32-48ac-9e71-c227455dfb4e None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:25:10.534589 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2c6a48-2a32-48ac-9e71-c227455dfb4e None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:25:10.534720 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2c6a48-2a32-48ac-9e71-c227455dfb4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:25:10.552724 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-0d2c6a48-2a32-48ac-9e71-c227455dfb4e None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:25:10.553919 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-0d2c6a48-2a32-48ac-9e71-c227455dfb4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:25:10.555808 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:25:10.619062 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 582/1155] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:25:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 08:25:10.642893 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abc45a5-4ffb-442e-aa4c-b46642394b5a None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:25:10.673459 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:25:10.742343 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abc45a5-4ffb-442e-aa4c-b46642394b5a None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:25:10.775430 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e23bb74-d681-44ed-b3c6-ce57029b9367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:25:10.776032 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e23bb74-d681-44ed-b3c6-ce57029b9367 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:25:10.776307 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e23bb74-d681-44ed-b3c6-ce57029b9367 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:25:10.776573 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e23bb74-d681-44ed-b3c6-ce57029b9367 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:25:10.786779 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e23bb74-d681-44ed-b3c6-ce57029b9367 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 08:25:10.791373 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-4e23bb74-d681-44ed-b3c6-ce57029b9367 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 08:25:10.791373 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:25:10.791373 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:25:10.791373 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:25:10.791373 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:25:10.791373 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:25:10.791373 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:25:10.791373 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:25:10.791373 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:25:10.791373 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:25:10.791373 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:25:10.791373 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:25:10.791373 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 08:25:10.791373 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 08:25:10.791373 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 08:25:10.791373 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 08:25:10.792311 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 08:25:10.792311 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 08:25:10.792311 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 08:25:10.792311 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 08:25:10.792311 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:25:10.792311 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:25:10.792311 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:25:10.792311 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:25:10.792311 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:25:10.792311 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:25:10.792311 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:25:10.792311 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:25:10.792311 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:25:10.792311 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:25:10.792311 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:25:10.792311 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:25:10.793234 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:25:10.793234 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:25:10.793234 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:25:10.793234 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:25:10.793234 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 08:25:10.793234 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 08:25:10.793234 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 08:25:10.793234 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 08:25:10.793234 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 08:25:10.793234 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 08:25:10.793234 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 08:25:10.793234 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:25:10.793234 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 574/1156] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Aug 10 08:25:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:25:10.798470 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2c6a48-2a32-48ac-9e71-c227455dfb4e None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:25:10.893492 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2c6a48-2a32-48ac-9e71-c227455dfb4e None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:25:10.925530 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14796fea-5bf8-4319-8828-5de5c105921f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:25:10.926064 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14796fea-5bf8-4319-8828-5de5c105921f None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:25:10.926342 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14796fea-5bf8-4319-8828-5de5c105921f None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:25:10.926610 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14796fea-5bf8-4319-8828-5de5c105921f None admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:25:10.927263 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14796fea-5bf8-4319-8828-5de5c105921f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:25:10.927565 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14796fea-5bf8-4319-8828-5de5c105921f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:25:10.928933 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14796fea-5bf8-4319-8828-5de5c105921f None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:25:10.933709 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:25:10.937455 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 583/1157] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Aug 10 08:25:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:25:10.943190 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e23bb74-d681-44ed-b3c6-ce57029b9367 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:25:11.033212 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:25:11.042270 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e23bb74-d681-44ed-b3c6-ce57029b9367 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:25:11.076296 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ff02e0-0e9f-4551-8a60-e3b30e0e234c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:25:11.076924 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ff02e0-0e9f-4551-8a60-e3b30e0e234c None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:25:11.077261 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ff02e0-0e9f-4551-8a60-e3b30e0e234c None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:25:11.077561 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ff02e0-0e9f-4551-8a60-e3b30e0e234c None admin] PATH_INFO: `/v3/users/23c67fddcd10410a8fe50561653558ef` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:25:11.078648 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ff02e0-0e9f-4551-8a60-e3b30e0e234c None admin] RBAC: Authorizing `identity:get_user(user_id=23c67fddcd10410a8fe50561653558ef)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:25:11.080066 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ff02e0-0e9f-4551-8a60-e3b30e0e234c None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:25:11.081711 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 575/1158] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Aug 10 08:25:10 2022] GET /identity/v3/users/23c67fddcd10410a8fe50561653558ef => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:25:11.087840 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14796fea-5bf8-4319-8828-5de5c105921f None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:25:11.197759 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14796fea-5bf8-4319-8828-5de5c105921f None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:25:11.230912 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac4140f-d1ea-4624-b215-27f6115e18a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:25:11.231513 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac4140f-d1ea-4624-b215-27f6115e18a5 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:25:11.231786 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac4140f-d1ea-4624-b215-27f6115e18a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:25:11.232053 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac4140f-d1ea-4624-b215-27f6115e18a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:25:11.298828 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:25:11.301194 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac4140f-d1ea-4624-b215-27f6115e18a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:25:11.303223 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac4140f-d1ea-4624-b215-27f6115e18a5 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:25:11.404121 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 584/1159] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Aug 10 08:25:11 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 08:25:11.410669 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ff02e0-0e9f-4551-8a60-e3b30e0e234c None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:25:11.509264 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ff02e0-0e9f-4551-8a60-e3b30e0e234c None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:25:11.541518 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4affe0d1-140e-4d21-9721-c30027e155d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:25:11.543165 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4affe0d1-140e-4d21-9721-c30027e155d8 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:25:11.543165 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4affe0d1-140e-4d21-9721-c30027e155d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:25:11.543165 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4affe0d1-140e-4d21-9721-c30027e155d8 None admin] PATH_INFO: `/v3/projects/eb0de551a4aa4b6dad55975a1913c1b9` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:25:11.543820 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4affe0d1-140e-4d21-9721-c30027e155d8 None admin] RBAC: Authorizing `identity:get_project(project_id=eb0de551a4aa4b6dad55975a1913c1b9)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:25:11.545271 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4affe0d1-140e-4d21-9721-c30027e155d8 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:25:11.547042 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 576/1160] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Aug 10 08:25:11 2022] GET /identity/v3/projects/eb0de551a4aa4b6dad55975a1913c1b9 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:25:11.553151 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac4140f-d1ea-4624-b215-27f6115e18a5 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:25:11.645987 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:25:11.648799 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac4140f-d1ea-4624-b215-27f6115e18a5 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:25:11.681294 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0694458b-679e-424f-a997-229f1e8aed31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:25:11.681880 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0694458b-679e-424f-a997-229f1e8aed31 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:25:11.682156 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0694458b-679e-424f-a997-229f1e8aed31 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:25:11.682430 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0694458b-679e-424f-a997-229f1e8aed31 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:25:11.683141 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0694458b-679e-424f-a997-229f1e8aed31 None admin] RBAC: Adding query filter params (role.id=ec3751692d9c4ecf81b4a71c936c46be, scope.project.id=eb0de551a4aa4b6dad55975a1913c1b9, user.id=23c67fddcd10410a8fe50561653558ef) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:25:11.683446 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0694458b-679e-424f-a997-229f1e8aed31 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:25:11.684856 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0694458b-679e-424f-a997-229f1e8aed31 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:25:11.690928 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 585/1161] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Aug 10 08:25:11 2022] GET /identity/v3/role_assignments?user.id=23c67fddcd10410a8fe50561653558ef&scope.project.id=eb0de551a4aa4b6dad55975a1913c1b9&role.id=ec3751692d9c4ecf81b4a71c936c46be => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:25:13.716226 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa2e9cf-86dc-4f0c-a937-a29b1e481d4b None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:25:13.716743 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa2e9cf-86dc-4f0c-a937-a29b1e481d4b None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:25:13.716868 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa2e9cf-86dc-4f0c-a937-a29b1e481d4b None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:25:13.718146 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 577/1162] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:25:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:25:13.723203 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e732ff-3efc-460e-9d8b-f0ab4918688d None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:25:13.723640 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e732ff-3efc-460e-9d8b-f0ab4918688d None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:25:13.723744 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e732ff-3efc-460e-9d8b-f0ab4918688d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:25:13.741433 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-a9e732ff-3efc-460e-9d8b-f0ab4918688d None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:25:13.742619 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-a9e732ff-3efc-460e-9d8b-f0ab4918688d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:25:13.807615 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 586/1163] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:25:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 08:25:13.831235 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa2e9cf-86dc-4f0c-a937-a29b1e481d4b None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:25:13.858168 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:25:13.930597 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa2e9cf-86dc-4f0c-a937-a29b1e481d4b None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:25:13.963991 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4507d738-5131-49f2-ad58-2aee1943ac25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:25:13.964576 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4507d738-5131-49f2-ad58-2aee1943ac25 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:25:13.964855 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4507d738-5131-49f2-ad58-2aee1943ac25 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:25:13.965149 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4507d738-5131-49f2-ad58-2aee1943ac25 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:25:14.029531 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4507d738-5131-49f2-ad58-2aee1943ac25 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:25:14.031403 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4507d738-5131-49f2-ad58-2aee1943ac25 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:25:14.057398 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:25:14.133638 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 578/1164] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Aug 10 08:25:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 08:25:14.140677 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e732ff-3efc-460e-9d8b-f0ab4918688d None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:25:14.229972 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:25:14.236801 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e732ff-3efc-460e-9d8b-f0ab4918688d None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:25:14.270184 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e8c356-5898-41ef-92d3-c965a933421a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:25:14.270794 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e8c356-5898-41ef-92d3-c965a933421a None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:25:14.271067 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e8c356-5898-41ef-92d3-c965a933421a None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:25:14.271331 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e8c356-5898-41ef-92d3-c965a933421a None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:25:14.277661 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e8c356-5898-41ef-92d3-c965a933421a None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:25:14.279083 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e8c356-5898-41ef-92d3-c965a933421a None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:25:14.290973 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-90e8c356-5898-41ef-92d3-c965a933421a None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 10 08:25:14.290973 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:25:14.290973 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 10 08:25:14.290973 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 10 08:25:14.290973 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 08:25:14.290973 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 08:25:14.290973 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 08:25:14.290973 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 08:25:14.290973 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 10 08:25:14.290973 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:25:14.290973 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 08:25:14.290973 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:25:14.290973 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:25:14.290973 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:25:14.290973 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:25:14.290973 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:25:14.290973 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:25:14.292143 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:25:14.292143 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:25:14.292143 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:25:14.292143 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:25:14.292143 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:25:14.292143 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:25:14.292143 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 08:25:14.292143 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 08:25:14.292143 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 08:25:14.292143 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 08:25:14.292143 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:25:14.292143 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:25:14.292143 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:25:14.292143 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:25:14.292143 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:25:14.292143 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:25:14.293050 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:25:14.293050 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:25:14.293050 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:25:14.293050 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:25:14.293050 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:25:14.293050 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:25:14.293050 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:25:14.293050 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:25:14.293050 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:25:14.293050 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:25:14.293050 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 10 08:25:14.293050 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 08:25:14.293050 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 10 08:25:14.293050 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:25:14.293050 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 587/1165] 10.222.0.57 () {56 vars in 1117 bytes} [Wed Aug 10 08:25:14 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 08:25:14.298185 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4507d738-5131-49f2-ad58-2aee1943ac25 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:25:14.394671 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4507d738-5131-49f2-ad58-2aee1943ac25 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:25:14.419940 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:25:14.428297 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ac96b6-6c9f-44bf-b0db-c016cf24dba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:25:14.428942 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ac96b6-6c9f-44bf-b0db-c016cf24dba8 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:25:14.429233 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ac96b6-6c9f-44bf-b0db-c016cf24dba8 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:25:14.429904 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ac96b6-6c9f-44bf-b0db-c016cf24dba8 None admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:25:14.430237 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ac96b6-6c9f-44bf-b0db-c016cf24dba8 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:25:14.430543 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ac96b6-6c9f-44bf-b0db-c016cf24dba8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:25:14.431893 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ac96b6-6c9f-44bf-b0db-c016cf24dba8 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:25:14.439634 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 579/1166] 10.222.0.57 () {56 vars in 1115 bytes} [Wed Aug 10 08:25:14 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:25:16.431464 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17005087-bbc8-4459-8a7f-1c2a586ae3cd None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:25:16.431931 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17005087-bbc8-4459-8a7f-1c2a586ae3cd None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:25:16.432045 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17005087-bbc8-4459-8a7f-1c2a586ae3cd None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:25:16.433438 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 588/1167] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:25:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:25:16.439502 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b5ba12-0193-4835-b732-2384ee38f542 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:25:16.439930 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b5ba12-0193-4835-b732-2384ee38f542 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:25:16.440044 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b5ba12-0193-4835-b732-2384ee38f542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:25:16.457632 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-38b5ba12-0193-4835-b732-2384ee38f542 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:25:16.458801 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-38b5ba12-0193-4835-b732-2384ee38f542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:25:16.523964 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 580/1168] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:25:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 08:25:16.546951 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17005087-bbc8-4459-8a7f-1c2a586ae3cd None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:25:16.643438 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17005087-bbc8-4459-8a7f-1c2a586ae3cd None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:25:16.676165 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca37b65-0bad-4cbf-bf43-25dfb8dd1075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:25:16.676751 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca37b65-0bad-4cbf-bf43-25dfb8dd1075 None admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:25:16.677053 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca37b65-0bad-4cbf-bf43-25dfb8dd1075 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:25:16.677321 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca37b65-0bad-4cbf-bf43-25dfb8dd1075 None admin] PATH_INFO: `/v3/domains` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:25:16.677968 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca37b65-0bad-4cbf-bf43-25dfb8dd1075 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:25:16.679309 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca37b65-0bad-4cbf-bf43-25dfb8dd1075 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:25:16.746610 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 589/1169] 10.222.0.57 () {60 vars in 1138 bytes} [Wed Aug 10 08:25:16 2022] POST /identity/v3/domains => generated 267 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:25:18.766416 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031a5763-2e96-434f-8b4a-9f34185375a3 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:25:18.766945 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031a5763-2e96-434f-8b4a-9f34185375a3 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:25:18.766945 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031a5763-2e96-434f-8b4a-9f34185375a3 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:25:18.768297 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 581/1170] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:25:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:25:18.773387 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736bbbe0-59d8-4a4d-a41e-ee383a502d39 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:25:18.773821 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736bbbe0-59d8-4a4d-a41e-ee383a502d39 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:25:18.773925 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736bbbe0-59d8-4a4d-a41e-ee383a502d39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:25:18.791362 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-736bbbe0-59d8-4a4d-a41e-ee383a502d39 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:25:18.792523 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-736bbbe0-59d8-4a4d-a41e-ee383a502d39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:25:18.801923 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:25:18.857112 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 590/1171] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:25:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 08:25:18.881269 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031a5763-2e96-434f-8b4a-9f34185375a3 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:25:18.977851 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031a5763-2e96-434f-8b4a-9f34185375a3 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:25:18.997667 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:25:19.011292 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb34d73-2225-415f-a93b-a1dd7b656a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:25:19.011909 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb34d73-2225-415f-a93b-a1dd7b656a81 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:25:19.012186 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb34d73-2225-415f-a93b-a1dd7b656a81 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:25:19.012455 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb34d73-2225-415f-a93b-a1dd7b656a81 None admin] PATH_INFO: `/v3/domains/35042d40da9648b7969abbd25834fee8` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:25:19.022796 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb34d73-2225-415f-a93b-a1dd7b656a81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=35042d40da9648b7969abbd25834fee8)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:25:19.024169 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb34d73-2225-415f-a93b-a1dd7b656a81 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:25:19.025773 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 582/1172] 10.222.0.57 () {56 vars in 1183 bytes} [Wed Aug 10 08:25:18 2022] GET /identity/v3/domains/35042d40da9648b7969abbd25834fee8 => generated 267 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:25:19.032559 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736bbbe0-59d8-4a4d-a41e-ee383a502d39 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:25:19.128613 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736bbbe0-59d8-4a4d-a41e-ee383a502d39 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:25:19.161147 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4541b0a9-8521-4761-86b9-65c5aec3ab65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:25:19.161743 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4541b0a9-8521-4761-86b9-65c5aec3ab65 None admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:25:19.162023 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4541b0a9-8521-4761-86b9-65c5aec3ab65 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:25:19.162293 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4541b0a9-8521-4761-86b9-65c5aec3ab65 None admin] PATH_INFO: `/v3/projects` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:25:19.162933 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4541b0a9-8521-4761-86b9-65c5aec3ab65 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:25:19.164361 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4541b0a9-8521-4761-86b9-65c5aec3ab65 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:25:19.179546 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:25:19.244035 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 591/1173] 10.222.0.57 () {60 vars in 1141 bytes} [Wed Aug 10 08:25:19 2022] POST /identity/v3/projects => generated 361 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:25:21.295603 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1521d394-688d-48c8-b2f3-412531dc0bcc None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:25:21.296145 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1521d394-688d-48c8-b2f3-412531dc0bcc None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:25:21.296145 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1521d394-688d-48c8-b2f3-412531dc0bcc None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:25:21.297320 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 583/1174] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:25:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:25:21.302497 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdef30b4-8efc-496f-b23e-a639ad763ccc None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:25:21.302898 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdef30b4-8efc-496f-b23e-a639ad763ccc None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:25:21.303043 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdef30b4-8efc-496f-b23e-a639ad763ccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:25:21.321146 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-bdef30b4-8efc-496f-b23e-a639ad763ccc None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:25:21.322306 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-bdef30b4-8efc-496f-b23e-a639ad763ccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:25:21.385945 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 592/1175] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:25:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 08:25:21.410804 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1521d394-688d-48c8-b2f3-412531dc0bcc None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:25:21.511787 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1521d394-688d-48c8-b2f3-412531dc0bcc None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:25:21.544184 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9373fc-d2b8-4ed3-b70f-ea485d27b020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:25:21.544786 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9373fc-d2b8-4ed3-b70f-ea485d27b020 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:25:21.545086 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9373fc-d2b8-4ed3-b70f-ea485d27b020 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:25:21.546360 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9373fc-d2b8-4ed3-b70f-ea485d27b020 None admin] PATH_INFO: `/v3/domains/35042d40da9648b7969abbd25834fee8` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:25:21.547401 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9373fc-d2b8-4ed3-b70f-ea485d27b020 None admin] RBAC: Authorizing `identity:get_domain(domain_id=35042d40da9648b7969abbd25834fee8)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:25:21.548813 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9373fc-d2b8-4ed3-b70f-ea485d27b020 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:25:21.550378 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 584/1176] 10.222.0.57 () {56 vars in 1183 bytes} [Wed Aug 10 08:25:21 2022] GET /identity/v3/domains/35042d40da9648b7969abbd25834fee8 => generated 267 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:25:21.556062 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdef30b4-8efc-496f-b23e-a639ad763ccc None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:25:21.651671 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdef30b4-8efc-496f-b23e-a639ad763ccc None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:25:21.676077 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:25:21.684197 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722c791c-2d5c-4e98-b473-7d50791d1e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:25:21.684782 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722c791c-2d5c-4e98-b473-7d50791d1e62 None admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:25:21.685088 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722c791c-2d5c-4e98-b473-7d50791d1e62 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:25:21.685357 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722c791c-2d5c-4e98-b473-7d50791d1e62 None admin] PATH_INFO: `/v3/users` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:25:21.686643 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722c791c-2d5c-4e98-b473-7d50791d1e62 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:25:21.694080 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722c791c-2d5c-4e98-b473-7d50791d1e62 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:25:21.797171 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 593/1177] 10.222.0.57 () {60 vars in 1132 bytes} [Wed Aug 10 08:25:21 2022] POST /identity/v3/users => generated 311 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 08:25:23.830944 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5b5dc5-98d8-486b-9213-efc54ebfabe0 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:25:23.831528 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5b5dc5-98d8-486b-9213-efc54ebfabe0 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:25:23.831528 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5b5dc5-98d8-486b-9213-efc54ebfabe0 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:25:23.832833 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 585/1178] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:25:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:25:23.838027 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6356b38d-368b-4c88-80e5-0a3172dfdb89 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:25:23.838431 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6356b38d-368b-4c88-80e5-0a3172dfdb89 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:25:23.838585 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6356b38d-368b-4c88-80e5-0a3172dfdb89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:25:23.856931 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-6356b38d-368b-4c88-80e5-0a3172dfdb89 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:25:23.858262 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-6356b38d-368b-4c88-80e5-0a3172dfdb89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:25:23.922658 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 594/1179] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:25:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 08:25:23.945631 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5b5dc5-98d8-486b-9213-efc54ebfabe0 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:25:23.995640 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:25:24.046018 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5b5dc5-98d8-486b-9213-efc54ebfabe0 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:25:24.078195 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20a0474-8824-4389-b180-35c81947e523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:25:24.078807 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20a0474-8824-4389-b180-35c81947e523 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:25:24.079084 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20a0474-8824-4389-b180-35c81947e523 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:25:24.079353 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20a0474-8824-4389-b180-35c81947e523 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:25:24.090239 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20a0474-8824-4389-b180-35c81947e523 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 08:25:24.095474 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-f20a0474-8824-4389-b180-35c81947e523 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 08:25:24.095474 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:25:24.095474 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:25:24.095474 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:25:24.095474 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:25:24.095474 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:25:24.095474 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:25:24.095474 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:25:24.095474 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:25:24.095474 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:25:24.095474 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:25:24.095474 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:25:24.095474 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 08:25:24.095474 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 08:25:24.095474 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 08:25:24.095474 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 08:25:24.096525 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 08:25:24.096525 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 08:25:24.096525 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 08:25:24.096525 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 08:25:24.096525 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:25:24.096525 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:25:24.096525 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:25:24.096525 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:25:24.096525 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:25:24.096525 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:25:24.096525 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:25:24.096525 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:25:24.096525 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:25:24.096525 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:25:24.096525 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:25:24.096525 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:25:24.097718 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:25:24.097718 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:25:24.097718 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:25:24.097718 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:25:24.097718 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 08:25:24.097718 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 08:25:24.097718 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 08:25:24.097718 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 08:25:24.097718 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 08:25:24.097718 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 08:25:24.097718 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 08:25:24.097718 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:25:24.097718 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 586/1180] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Aug 10 08:25:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:25:24.102516 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6356b38d-368b-4c88-80e5-0a3172dfdb89 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:25:24.198981 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6356b38d-368b-4c88-80e5-0a3172dfdb89 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:25:24.211322 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:25:24.232095 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13041e76-0d54-4302-a450-2d2973917280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:25:24.233257 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13041e76-0d54-4302-a450-2d2973917280 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:25:24.233536 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13041e76-0d54-4302-a450-2d2973917280 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:25:24.233802 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13041e76-0d54-4302-a450-2d2973917280 None admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:25:24.234448 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13041e76-0d54-4302-a450-2d2973917280 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:25:24.234753 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13041e76-0d54-4302-a450-2d2973917280 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:25:24.236136 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13041e76-0d54-4302-a450-2d2973917280 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:25:24.244020 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 595/1181] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Aug 10 08:25:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:25:24.250110 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20a0474-8824-4389-b180-35c81947e523 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:25:24.347234 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20a0474-8824-4389-b180-35c81947e523 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:25:24.355477 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:25:24.380092 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa31eb0-f059-4d53-81b8-af0b4d83064f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:25:24.380660 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa31eb0-f059-4d53-81b8-af0b4d83064f None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:25:24.380955 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa31eb0-f059-4d53-81b8-af0b4d83064f None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:25:24.381227 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa31eb0-f059-4d53-81b8-af0b4d83064f None admin] PATH_INFO: `/v3/users/f216225334c944ad8ca379e19e13a90a` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:25:24.395260 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa31eb0-f059-4d53-81b8-af0b4d83064f None admin] RBAC: Authorizing `identity:get_user(user_id=f216225334c944ad8ca379e19e13a90a)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:25:24.396723 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa31eb0-f059-4d53-81b8-af0b4d83064f None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:25:24.398432 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 587/1182] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Aug 10 08:25:24 2022] GET /identity/v3/users/f216225334c944ad8ca379e19e13a90a => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:25:24.404134 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13041e76-0d54-4302-a450-2d2973917280 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:25:24.503354 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13041e76-0d54-4302-a450-2d2973917280 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:25:24.536168 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807d31aa-57e7-4fab-99eb-bdc294e8913d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:25:24.539248 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807d31aa-57e7-4fab-99eb-bdc294e8913d None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:25:24.539248 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807d31aa-57e7-4fab-99eb-bdc294e8913d None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:25:24.539248 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807d31aa-57e7-4fab-99eb-bdc294e8913d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:25:24.578791 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:25:24.603008 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807d31aa-57e7-4fab-99eb-bdc294e8913d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:25:24.604383 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807d31aa-57e7-4fab-99eb-bdc294e8913d None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:25:24.698909 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 596/1183] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Aug 10 08:25:24 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 08:25:24.705346 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa31eb0-f059-4d53-81b8-af0b4d83064f None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:25:24.803386 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa31eb0-f059-4d53-81b8-af0b4d83064f None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:25:24.836265 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7309b83-3c7f-43f4-93d6-065f99251e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:25:24.836853 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7309b83-3c7f-43f4-93d6-065f99251e2c None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:25:24.837158 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7309b83-3c7f-43f4-93d6-065f99251e2c None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:25:24.837425 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7309b83-3c7f-43f4-93d6-065f99251e2c None admin] PATH_INFO: `/v3/projects/23eb38f78a924d00966f4670abd6555d` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:25:24.838512 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7309b83-3c7f-43f4-93d6-065f99251e2c None admin] RBAC: Authorizing `identity:get_project(project_id=23eb38f78a924d00966f4670abd6555d)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:25:24.839919 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7309b83-3c7f-43f4-93d6-065f99251e2c None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:25:24.841546 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 588/1184] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Aug 10 08:25:24 2022] GET /identity/v3/projects/23eb38f78a924d00966f4670abd6555d => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:25:24.848206 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807d31aa-57e7-4fab-99eb-bdc294e8913d None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:25:24.919605 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:25:24.943517 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807d31aa-57e7-4fab-99eb-bdc294e8913d None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:25:24.977266 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80459e08-3b65-4ea7-9455-963f17bff75e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:25:24.977856 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80459e08-3b65-4ea7-9455-963f17bff75e None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:25:24.978131 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80459e08-3b65-4ea7-9455-963f17bff75e None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:25:24.978397 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80459e08-3b65-4ea7-9455-963f17bff75e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:25:24.979106 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80459e08-3b65-4ea7-9455-963f17bff75e None admin] RBAC: Adding query filter params (role.id=ec3751692d9c4ecf81b4a71c936c46be, scope.project.id=23eb38f78a924d00966f4670abd6555d, user.id=f216225334c944ad8ca379e19e13a90a) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:25:24.979409 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80459e08-3b65-4ea7-9455-963f17bff75e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:25:24.980810 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80459e08-3b65-4ea7-9455-963f17bff75e None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:25:24.986050 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 597/1185] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Aug 10 08:25:24 2022] GET /identity/v3/role_assignments?user.id=f216225334c944ad8ca379e19e13a90a&scope.project.id=23eb38f78a924d00966f4670abd6555d&role.id=ec3751692d9c4ecf81b4a71c936c46be => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:25:26.937651 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e094cdfa-9284-4008-ad51-4a67ef02936d None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:25:26.938208 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e094cdfa-9284-4008-ad51-4a67ef02936d None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:25:26.938208 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e094cdfa-9284-4008-ad51-4a67ef02936d None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:25:26.939532 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 589/1186] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:25:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:25:26.944680 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f082e125-82b9-4e96-98a7-d1da0cd666f1 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:25:26.945152 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f082e125-82b9-4e96-98a7-d1da0cd666f1 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:25:26.945246 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f082e125-82b9-4e96-98a7-d1da0cd666f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:25:26.962735 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-f082e125-82b9-4e96-98a7-d1da0cd666f1 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:25:26.963907 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-f082e125-82b9-4e96-98a7-d1da0cd666f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:25:27.028337 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 598/1187] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:25:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 08:25:27.050418 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e094cdfa-9284-4008-ad51-4a67ef02936d None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:25:27.084462 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:25:27.148639 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e094cdfa-9284-4008-ad51-4a67ef02936d None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:25:27.181601 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602d3c7e-7ef9-4698-b2f2-c318df2d6285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:25:27.182205 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602d3c7e-7ef9-4698-b2f2-c318df2d6285 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:25:27.182479 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602d3c7e-7ef9-4698-b2f2-c318df2d6285 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:25:27.182741 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602d3c7e-7ef9-4698-b2f2-c318df2d6285 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:25:27.192517 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602d3c7e-7ef9-4698-b2f2-c318df2d6285 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 08:25:27.197048 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-602d3c7e-7ef9-4698-b2f2-c318df2d6285 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 08:25:27.197048 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:25:27.197048 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:25:27.197048 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:25:27.197048 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:25:27.197048 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:25:27.197048 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:25:27.197048 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:25:27.197048 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:25:27.197048 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:25:27.197048 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:25:27.197048 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:25:27.197048 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 08:25:27.197048 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 08:25:27.197048 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 08:25:27.197048 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 08:25:27.198194 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 08:25:27.198194 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 08:25:27.198194 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 08:25:27.198194 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 08:25:27.198194 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:25:27.198194 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:25:27.198194 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:25:27.198194 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:25:27.198194 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:25:27.198194 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:25:27.198194 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:25:27.198194 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:25:27.198194 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:25:27.198194 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:25:27.198194 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:25:27.198194 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:25:27.199434 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:25:27.199434 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:25:27.199434 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:25:27.199434 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:25:27.199434 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 08:25:27.199434 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 08:25:27.199434 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 08:25:27.199434 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 08:25:27.199434 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 08:25:27.199434 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 08:25:27.199434 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 08:25:27.199434 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:25:27.199434 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 590/1188] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Aug 10 08:25:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:25:27.204125 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f082e125-82b9-4e96-98a7-d1da0cd666f1 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:25:27.269261 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:25:27.301987 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f082e125-82b9-4e96-98a7-d1da0cd666f1 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:25:27.334446 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ef9e75-1ff3-4da8-b03b-17d71ca820a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:25:27.335074 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ef9e75-1ff3-4da8-b03b-17d71ca820a7 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:25:27.335352 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ef9e75-1ff3-4da8-b03b-17d71ca820a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:25:27.335621 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ef9e75-1ff3-4da8-b03b-17d71ca820a7 None admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:25:27.336279 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ef9e75-1ff3-4da8-b03b-17d71ca820a7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:25:27.336585 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ef9e75-1ff3-4da8-b03b-17d71ca820a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:25:27.337986 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ef9e75-1ff3-4da8-b03b-17d71ca820a7 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:25:27.346257 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 599/1189] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Aug 10 08:25:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:25:27.352007 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602d3c7e-7ef9-4698-b2f2-c318df2d6285 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:25:27.445474 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:25:27.452285 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602d3c7e-7ef9-4698-b2f2-c318df2d6285 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:25:27.487034 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987e87b6-0c4b-482d-88d1-11503f5512d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:25:27.487652 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987e87b6-0c4b-482d-88d1-11503f5512d9 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:25:27.487927 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987e87b6-0c4b-482d-88d1-11503f5512d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:25:27.488193 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987e87b6-0c4b-482d-88d1-11503f5512d9 None admin] PATH_INFO: `/v3/users/f216225334c944ad8ca379e19e13a90a` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:25:27.489329 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987e87b6-0c4b-482d-88d1-11503f5512d9 None admin] RBAC: Authorizing `identity:get_user(user_id=f216225334c944ad8ca379e19e13a90a)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:25:27.490740 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987e87b6-0c4b-482d-88d1-11503f5512d9 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:25:27.492304 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 591/1190] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Aug 10 08:25:27 2022] GET /identity/v3/users/f216225334c944ad8ca379e19e13a90a => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:25:27.498501 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ef9e75-1ff3-4da8-b03b-17d71ca820a7 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:25:27.595327 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ef9e75-1ff3-4da8-b03b-17d71ca820a7 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:25:27.622441 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:25:27.627964 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0e16f8-b5df-4336-a2c9-83ffcd8d1ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:25:27.628584 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0e16f8-b5df-4336-a2c9-83ffcd8d1ce5 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:25:27.628862 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0e16f8-b5df-4336-a2c9-83ffcd8d1ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:25:27.629213 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0e16f8-b5df-4336-a2c9-83ffcd8d1ce5 None admin] PATH_INFO: `/v3/projects/23eb38f78a924d00966f4670abd6555d` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:25:27.630314 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0e16f8-b5df-4336-a2c9-83ffcd8d1ce5 None admin] RBAC: Authorizing `identity:get_project(project_id=23eb38f78a924d00966f4670abd6555d)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:25:27.631738 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0e16f8-b5df-4336-a2c9-83ffcd8d1ce5 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:25:27.633388 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 600/1191] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Aug 10 08:25:27 2022] GET /identity/v3/projects/23eb38f78a924d00966f4670abd6555d => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:25:27.639965 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987e87b6-0c4b-482d-88d1-11503f5512d9 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:25:27.737328 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987e87b6-0c4b-482d-88d1-11503f5512d9 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:25:27.770753 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3bcfcb-cfd9-4ee9-b87e-00d2fcd2365d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:25:27.771392 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3bcfcb-cfd9-4ee9-b87e-00d2fcd2365d None admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:25:27.771669 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3bcfcb-cfd9-4ee9-b87e-00d2fcd2365d None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:25:27.771936 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3bcfcb-cfd9-4ee9-b87e-00d2fcd2365d None admin] PATH_INFO: `/v3/projects/23eb38f78a924d00966f4670abd6555d/users/f216225334c944ad8ca379e19e13a90a/roles/ec3751692d9c4ecf81b4a71c936c46be` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:25:27.773817 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3bcfcb-cfd9-4ee9-b87e-00d2fcd2365d None admin] RBAC: Authorizing `identity:create_grant(project_id=23eb38f78a924d00966f4670abd6555d, user_id=f216225334c944ad8ca379e19e13a90a, role_id=ec3751692d9c4ecf81b4a71c936c46be)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:25:27.775232 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3bcfcb-cfd9-4ee9-b87e-00d2fcd2365d None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:25:27.842611 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 592/1192] 10.222.0.57 () {58 vars in 1439 bytes} [Wed Aug 10 08:25:27 2022] PUT /identity/v3/projects/23eb38f78a924d00966f4670abd6555d/users/f216225334c944ad8ca379e19e13a90a/roles/ec3751692d9c4ecf81b4a71c936c46be => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:25:29.819509 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd0f975-80fb-4b64-8994-3d3300dfecd5 None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:25:29.820061 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd0f975-80fb-4b64-8994-3d3300dfecd5 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:25:29.820061 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd0f975-80fb-4b64-8994-3d3300dfecd5 None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:25:29.821429 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 601/1193] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:25:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:25:29.826958 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3a2e37-113b-491f-b2b8-f2cb32db8015 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:25:29.827341 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3a2e37-113b-491f-b2b8-f2cb32db8015 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:25:29.827492 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3a2e37-113b-491f-b2b8-f2cb32db8015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:25:29.845491 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-1d3a2e37-113b-491f-b2b8-f2cb32db8015 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:25:29.846676 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-1d3a2e37-113b-491f-b2b8-f2cb32db8015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:25:29.864757 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:25:29.915220 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 593/1194] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:25:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 08:25:29.939902 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd0f975-80fb-4b64-8994-3d3300dfecd5 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:25:30.037451 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd0f975-80fb-4b64-8994-3d3300dfecd5 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:25:30.069864 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cd7db1-3b22-47b1-8b18-0fe362c4c973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:25:30.070413 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cd7db1-3b22-47b1-8b18-0fe362c4c973 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:25:30.070688 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cd7db1-3b22-47b1-8b18-0fe362c4c973 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:25:30.070954 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cd7db1-3b22-47b1-8b18-0fe362c4c973 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:25:30.081301 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cd7db1-3b22-47b1-8b18-0fe362c4c973 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 08:25:30.085802 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-a0cd7db1-3b22-47b1-8b18-0fe362c4c973 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 08:25:30.085802 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:25:30.085802 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:25:30.085802 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:25:30.085802 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:25:30.085802 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:25:30.085802 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:25:30.085802 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:25:30.085802 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:25:30.085802 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:25:30.085802 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:25:30.085802 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:25:30.085802 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 08:25:30.085802 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 08:25:30.085802 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 08:25:30.085802 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 08:25:30.086912 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 08:25:30.086912 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 08:25:30.086912 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 08:25:30.086912 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 08:25:30.086912 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:25:30.086912 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:25:30.086912 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:25:30.086912 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:25:30.086912 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:25:30.086912 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:25:30.086912 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:25:30.086912 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:25:30.086912 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:25:30.086912 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:25:30.086912 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:25:30.086912 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:25:30.087983 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:25:30.087983 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:25:30.087983 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:25:30.087983 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:25:30.087983 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 08:25:30.087983 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 08:25:30.087983 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 08:25:30.087983 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 08:25:30.087983 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 08:25:30.087983 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 08:25:30.087983 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 08:25:30.087983 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:25:30.087983 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 602/1195] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Aug 10 08:25:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:25:30.093001 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3a2e37-113b-491f-b2b8-f2cb32db8015 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:25:30.188548 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3a2e37-113b-491f-b2b8-f2cb32db8015 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:25:30.220795 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf7ad5a-ed15-4f21-8b0c-f97ec8587bd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:25:30.221406 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf7ad5a-ed15-4f21-8b0c-f97ec8587bd1 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:25:30.221679 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf7ad5a-ed15-4f21-8b0c-f97ec8587bd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:25:30.221945 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf7ad5a-ed15-4f21-8b0c-f97ec8587bd1 None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:25:30.222594 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf7ad5a-ed15-4f21-8b0c-f97ec8587bd1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:25:30.222891 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf7ad5a-ed15-4f21-8b0c-f97ec8587bd1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:25:30.224266 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf7ad5a-ed15-4f21-8b0c-f97ec8587bd1 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:25:30.231900 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 594/1196] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Aug 10 08:25:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:25:30.238353 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cd7db1-3b22-47b1-8b18-0fe362c4c973 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:25:30.295496 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:25:30.335847 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cd7db1-3b22-47b1-8b18-0fe362c4c973 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:25:30.368910 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56cd033-464a-4b3d-8cf3-3989cf09d0a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:25:30.369548 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56cd033-464a-4b3d-8cf3-3989cf09d0a9 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:25:30.369828 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56cd033-464a-4b3d-8cf3-3989cf09d0a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:25:30.373948 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56cd033-464a-4b3d-8cf3-3989cf09d0a9 None admin] PATH_INFO: `/v3/users/f216225334c944ad8ca379e19e13a90a` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:25:30.373948 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56cd033-464a-4b3d-8cf3-3989cf09d0a9 None admin] RBAC: Authorizing `identity:get_user(user_id=f216225334c944ad8ca379e19e13a90a)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:25:30.373948 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56cd033-464a-4b3d-8cf3-3989cf09d0a9 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:25:30.374723 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 603/1197] 10.222.0.57 () {56 vars in 1177 bytes} [Wed Aug 10 08:25:30 2022] GET /identity/v3/users/f216225334c944ad8ca379e19e13a90a => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:25:30.380486 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf7ad5a-ed15-4f21-8b0c-f97ec8587bd1 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:25:30.397281 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:25:30.479792 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf7ad5a-ed15-4f21-8b0c-f97ec8587bd1 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:25:30.512282 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b205dcf2-8a84-4073-8773-65471a37c135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:25:30.512902 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b205dcf2-8a84-4073-8773-65471a37c135 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:25:30.513209 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b205dcf2-8a84-4073-8773-65471a37c135 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:25:30.514122 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b205dcf2-8a84-4073-8773-65471a37c135 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:25:30.578624 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b205dcf2-8a84-4073-8773-65471a37c135 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:25:30.580050 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b205dcf2-8a84-4073-8773-65471a37c135 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:25:30.593698 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:25:30.674888 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 595/1198] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Aug 10 08:25:30 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 08:25:30.680779 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56cd033-464a-4b3d-8cf3-3989cf09d0a9 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:25:30.777120 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56cd033-464a-4b3d-8cf3-3989cf09d0a9 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:25:30.801317 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:25:30.809609 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c0a500-f9e3-4d4a-815e-01b78dc1908b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:25:30.810293 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c0a500-f9e3-4d4a-815e-01b78dc1908b None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:25:30.811336 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c0a500-f9e3-4d4a-815e-01b78dc1908b None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:25:30.811336 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c0a500-f9e3-4d4a-815e-01b78dc1908b None admin] PATH_INFO: `/v3/projects/23eb38f78a924d00966f4670abd6555d` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:25:30.812423 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c0a500-f9e3-4d4a-815e-01b78dc1908b None admin] RBAC: Authorizing `identity:get_project(project_id=23eb38f78a924d00966f4670abd6555d)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:25:30.813867 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c0a500-f9e3-4d4a-815e-01b78dc1908b None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:25:30.815469 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 604/1199] 10.222.0.57 () {56 vars in 1186 bytes} [Wed Aug 10 08:25:30 2022] GET /identity/v3/projects/23eb38f78a924d00966f4670abd6555d => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:25:30.821477 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b205dcf2-8a84-4073-8773-65471a37c135 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:25:30.916919 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b205dcf2-8a84-4073-8773-65471a37c135 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:25:30.930934 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:25:30.950445 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2003c6a-a7c9-4f53-b434-2013ddf276e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:25:30.951054 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2003c6a-a7c9-4f53-b434-2013ddf276e6 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:25:30.951423 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2003c6a-a7c9-4f53-b434-2013ddf276e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:25:30.951775 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2003c6a-a7c9-4f53-b434-2013ddf276e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:25:30.952735 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2003c6a-a7c9-4f53-b434-2013ddf276e6 None admin] RBAC: Adding query filter params (role.id=ec3751692d9c4ecf81b4a71c936c46be, scope.project.id=23eb38f78a924d00966f4670abd6555d, user.id=f216225334c944ad8ca379e19e13a90a) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:25:30.953075 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2003c6a-a7c9-4f53-b434-2013ddf276e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:25:30.954490 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2003c6a-a7c9-4f53-b434-2013ddf276e6 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:25:30.960029 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 596/1200] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Aug 10 08:25:30 2022] GET /identity/v3/role_assignments?user.id=f216225334c944ad8ca379e19e13a90a&scope.project.id=23eb38f78a924d00966f4670abd6555d&role.id=ec3751692d9c4ecf81b4a71c936c46be => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:25:40.836454 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=3 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:25:40.957511 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=11 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:26:42.718571 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d0f75d-c906-4bb0-ba9c-03ab2b10c4f4 None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:26:42.719154 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d0f75d-c906-4bb0-ba9c-03ab2b10c4f4 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:26:42.719154 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d0f75d-c906-4bb0-ba9c-03ab2b10c4f4 None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:26:42.720436 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 605/1201] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:26:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:26:42.725705 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42ad887-65eb-48d4-bcb9-91fdf24a11a2 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:26:42.726432 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42ad887-65eb-48d4-bcb9-91fdf24a11a2 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:26:42.726432 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42ad887-65eb-48d4-bcb9-91fdf24a11a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:26:42.745222 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-b42ad887-65eb-48d4-bcb9-91fdf24a11a2 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:26:42.746407 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-b42ad887-65eb-48d4-bcb9-91fdf24a11a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:26:42.813492 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 597/1202] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:26:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 08:26:42.838347 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d0f75d-c906-4bb0-ba9c-03ab2b10c4f4 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:26:42.940624 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d0f75d-c906-4bb0-ba9c-03ab2b10c4f4 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:26:42.972961 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f065717-bca9-42ac-9875-e8a8888b6f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:26:42.973519 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f065717-bca9-42ac-9875-e8a8888b6f0d None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:26:42.973791 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f065717-bca9-42ac-9875-e8a8888b6f0d None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:26:42.974054 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f065717-bca9-42ac-9875-e8a8888b6f0d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:26:42.984344 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f065717-bca9-42ac-9875-e8a8888b6f0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:26:42.985824 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f065717-bca9-42ac-9875-e8a8888b6f0d None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:26:43.000609 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-7f065717-bca9-42ac-9875-e8a8888b6f0d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:26:43.000609 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:26:43.000609 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:26:43.000609 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:26:43.000609 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:26:43.000609 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:26:43.000609 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:26:43.000609 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:26:43.000609 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:26:43.000609 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:26:43.000609 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:26:43.000609 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:26:43.000609 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 08:26:43.000609 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 08:26:43.000609 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 08:26:43.000609 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 08:26:43.006858 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:26:43.006858 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:26:43.006858 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:26:43.006858 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:26:43.006858 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:26:43.006858 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:26:43.006858 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:26:43.006858 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:26:43.006858 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:26:43.006858 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:26:43.006858 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:26:43.006858 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:26:43.006858 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:26:43.006858 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:26:43.006858 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:26:43.006858 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:26:43.006858 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 08:26:43.008091 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 08:26:43.008091 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:26:43.008091 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:26:43.008091 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 606/1203] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:26:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:26:43.008397 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42ad887-65eb-48d4-bcb9-91fdf24a11a2 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:26:43.105076 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42ad887-65eb-48d4-bcb9-91fdf24a11a2 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:26:43.133346 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:26:43.139520 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d4bc48-a55c-4113-89fb-9f18879df03a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:26:43.140121 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d4bc48-a55c-4113-89fb-9f18879df03a None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:26:43.140517 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d4bc48-a55c-4113-89fb-9f18879df03a None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:26:43.140699 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d4bc48-a55c-4113-89fb-9f18879df03a None admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:26:43.141366 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d4bc48-a55c-4113-89fb-9f18879df03a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:26:43.141672 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d4bc48-a55c-4113-89fb-9f18879df03a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:26:43.143030 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d4bc48-a55c-4113-89fb-9f18879df03a None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:26:43.154408 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 598/1204] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Aug 10 08:26:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:26:43.160253 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f065717-bca9-42ac-9875-e8a8888b6f0d None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:26:43.213326 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:26:43.260734 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f065717-bca9-42ac-9875-e8a8888b6f0d None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:26:43.291806 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9fba33-c50d-4832-9679-34f5b3b11c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:26:43.292407 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9fba33-c50d-4832-9679-34f5b3b11c88 None admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:26:43.292687 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9fba33-c50d-4832-9679-34f5b3b11c88 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:26:43.292993 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9fba33-c50d-4832-9679-34f5b3b11c88 None admin] PATH_INFO: `/v3/users` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:26:43.293629 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9fba33-c50d-4832-9679-34f5b3b11c88 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:26:43.295754 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9fba33-c50d-4832-9679-34f5b3b11c88 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:26:43.411054 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 607/1205] 10.222.0.57 () {60 vars in 1132 bytes} [Wed Aug 10 08:26:43 2022] POST /identity/v3/users => generated 251 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:26:45.456164 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e9587d-7bc2-4bcc-8343-459f28ec92b2 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:26:45.456685 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e9587d-7bc2-4bcc-8343-459f28ec92b2 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:26:45.456685 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e9587d-7bc2-4bcc-8343-459f28ec92b2 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:26:45.458072 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 599/1206] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:26:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:26:45.463052 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edf06c4-20d6-4e52-89d4-4f71bb6124bf None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:26:45.463489 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edf06c4-20d6-4e52-89d4-4f71bb6124bf None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:26:45.463584 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edf06c4-20d6-4e52-89d4-4f71bb6124bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:26:45.482770 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-1edf06c4-20d6-4e52-89d4-4f71bb6124bf None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:26:45.483950 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-1edf06c4-20d6-4e52-89d4-4f71bb6124bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:26:45.550437 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 608/1207] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:26:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 08:26:45.574170 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e9587d-7bc2-4bcc-8343-459f28ec92b2 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:26:45.674090 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e9587d-7bc2-4bcc-8343-459f28ec92b2 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:26:45.706207 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac58350f-38ff-476f-874b-b11a41b54510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:26:45.706826 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac58350f-38ff-476f-874b-b11a41b54510 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:26:45.707100 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac58350f-38ff-476f-874b-b11a41b54510 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:26:45.707364 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac58350f-38ff-476f-874b-b11a41b54510 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:26:45.718022 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac58350f-38ff-476f-874b-b11a41b54510 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 08:26:45.722571 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-ac58350f-38ff-476f-874b-b11a41b54510 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 08:26:45.722571 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:26:45.722571 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:26:45.722571 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:26:45.722571 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:26:45.722571 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:26:45.722571 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:26:45.722571 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:26:45.722571 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:26:45.722571 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:26:45.722571 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:26:45.722571 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:26:45.722571 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 08:26:45.722571 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 08:26:45.722571 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 08:26:45.722571 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 08:26:45.723827 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 08:26:45.723827 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 08:26:45.723827 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 08:26:45.723827 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 08:26:45.723827 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:26:45.723827 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:26:45.723827 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:26:45.723827 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:26:45.723827 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:26:45.723827 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:26:45.723827 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:26:45.723827 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:26:45.723827 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:26:45.723827 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:26:45.723827 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:26:45.723827 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:26:45.724864 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:26:45.724864 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:26:45.724864 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:26:45.724864 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:26:45.724864 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 08:26:45.724864 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 08:26:45.724864 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 08:26:45.724864 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 08:26:45.724864 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 08:26:45.724864 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 08:26:45.724864 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 08:26:45.724864 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:26:45.724864 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 600/1208] 10.222.0.57 () {56 vars in 1102 bytes} [Wed Aug 10 08:26:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:26:45.729674 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edf06c4-20d6-4e52-89d4-4f71bb6124bf None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:26:45.769183 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:26:45.827507 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edf06c4-20d6-4e52-89d4-4f71bb6124bf None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:26:45.860736 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290a8942-d688-4f8f-b2c3-bd5552453f56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:26:45.861366 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290a8942-d688-4f8f-b2c3-bd5552453f56 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:26:45.861643 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290a8942-d688-4f8f-b2c3-bd5552453f56 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:26:45.861935 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290a8942-d688-4f8f-b2c3-bd5552453f56 None admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:26:45.862715 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290a8942-d688-4f8f-b2c3-bd5552453f56 None admin] RBAC: Adding query filter params (name=service) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:26:45.863021 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290a8942-d688-4f8f-b2c3-bd5552453f56 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:26:45.864371 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290a8942-d688-4f8f-b2c3-bd5552453f56 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:26:45.873066 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 609/1209] 10.222.0.57 () {56 vars in 1103 bytes} [Wed Aug 10 08:26:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:26:45.879997 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac58350f-38ff-476f-874b-b11a41b54510 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:26:45.930166 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:26:45.979259 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac58350f-38ff-476f-874b-b11a41b54510 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:26:46.013649 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cfb33c-a8ec-46c8-b8bd-7c06469a05b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:26:46.014343 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cfb33c-a8ec-46c8-b8bd-7c06469a05b7 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:26:46.014618 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cfb33c-a8ec-46c8-b8bd-7c06469a05b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:26:46.014888 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cfb33c-a8ec-46c8-b8bd-7c06469a05b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:26:46.025299 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cfb33c-a8ec-46c8-b8bd-7c06469a05b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:26:46.026826 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cfb33c-a8ec-46c8-b8bd-7c06469a05b7 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:26:46.041017 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-f2cfb33c-a8ec-46c8-b8bd-7c06469a05b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:26:46.041017 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:26:46.041017 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:26:46.041017 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:26:46.041017 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:26:46.041017 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:26:46.041017 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:26:46.041017 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:26:46.041017 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:26:46.041017 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:26:46.041017 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:26:46.041017 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:26:46.041017 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 08:26:46.041017 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 08:26:46.041017 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 08:26:46.041017 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 08:26:46.042156 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:26:46.042156 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:26:46.042156 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:26:46.042156 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:26:46.042156 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:26:46.042156 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:26:46.042156 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:26:46.042156 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:26:46.042156 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:26:46.042156 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:26:46.042156 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:26:46.042156 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:26:46.042156 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:26:46.042156 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:26:46.042156 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:26:46.042156 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:26:46.042156 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 08:26:46.043396 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 08:26:46.043396 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:26:46.043396 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:26:46.043396 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 601/1210] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:26:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:26:46.048076 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290a8942-d688-4f8f-b2c3-bd5552453f56 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:26:46.147953 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:26:46.150813 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290a8942-d688-4f8f-b2c3-bd5552453f56 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:26:46.183435 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d17c14-01c8-48c1-b64b-f3a1dae7a62f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:26:46.183976 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d17c14-01c8-48c1-b64b-f3a1dae7a62f None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:26:46.184255 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d17c14-01c8-48c1-b64b-f3a1dae7a62f None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:26:46.185067 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d17c14-01c8-48c1-b64b-f3a1dae7a62f None admin] PATH_INFO: `/v3/domains` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:26:46.185700 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d17c14-01c8-48c1-b64b-f3a1dae7a62f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:26:46.185999 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d17c14-01c8-48c1-b64b-f3a1dae7a62f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:26:46.187389 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d17c14-01c8-48c1-b64b-f3a1dae7a62f None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:26:46.199633 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 610/1211] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Aug 10 08:26:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:26:46.206060 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cfb33c-a8ec-46c8-b8bd-7c06469a05b7 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:26:46.306293 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cfb33c-a8ec-46c8-b8bd-7c06469a05b7 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:26:46.307853 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:26:46.340479 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812457e1-3862-4551-8e07-29bd38fced31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:26:46.341710 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812457e1-3862-4551-8e07-29bd38fced31 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:26:46.341986 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812457e1-3862-4551-8e07-29bd38fced31 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:26:46.342252 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812457e1-3862-4551-8e07-29bd38fced31 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:26:46.350832 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812457e1-3862-4551-8e07-29bd38fced31 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:26:46.352258 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812457e1-3862-4551-8e07-29bd38fced31 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:26:46.363778 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-812457e1-3862-4551-8e07-29bd38fced31 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 10 08:26:46.363778 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:26:46.363778 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:26:46.363778 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:26:46.363778 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:26:46.363778 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:26:46.363778 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:26:46.363778 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:26:46.363778 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:26:46.363778 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:26:46.363778 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:26:46.363778 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:26:46.363778 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 08:26:46.363778 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 08:26:46.363778 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 08:26:46.363778 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 08:26:46.369327 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:26:46.369327 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:26:46.369327 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 08:26:46.369327 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 08:26:46.369327 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 08:26:46.369327 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 08:26:46.369327 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:26:46.369327 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:26:46.369327 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:26:46.369327 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:26:46.369327 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:26:46.369327 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:26:46.369327 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:26:46.369327 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:26:46.369327 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:26:46.369327 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:26:46.369327 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:26:46.369327 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:26:46.370570 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:26:46.370570 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:26:46.370570 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 08:26:46.370570 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 08:26:46.370570 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 08:26:46.370570 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 08:26:46.370570 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 08:26:46.370570 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 08:26:46.370570 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 10 08:26:46.370570 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:26:46.370570 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 602/1212] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:26:46 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:26:46.371340 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d17c14-01c8-48c1-b64b-f3a1dae7a62f None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:26:46.475906 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d17c14-01c8-48c1-b64b-f3a1dae7a62f None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:26:46.512043 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a9f9f9-b442-4502-9bb4-2e77c8254833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:26:46.512694 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a9f9f9-b442-4502-9bb4-2e77c8254833 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:26:46.513150 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a9f9f9-b442-4502-9bb4-2e77c8254833 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:26:46.513303 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a9f9f9-b442-4502-9bb4-2e77c8254833 None admin] PATH_INFO: `/v3/users` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:26:46.514035 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a9f9f9-b442-4502-9bb4-2e77c8254833 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:26:46.514336 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a9f9f9-b442-4502-9bb4-2e77c8254833 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:26:46.515812 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a9f9f9-b442-4502-9bb4-2e77c8254833 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:26:46.533102 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 611/1213] 10.222.0.57 () {56 vars in 1143 bytes} [Wed Aug 10 08:26:46 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:26:46.538722 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812457e1-3862-4551-8e07-29bd38fced31 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:26:46.640547 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812457e1-3862-4551-8e07-29bd38fced31 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:26:46.675931 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71350a6-bc7f-4586-a2aa-de2b79fe3b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:26:46.676523 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71350a6-bc7f-4586-a2aa-de2b79fe3b61 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:26:46.676797 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71350a6-bc7f-4586-a2aa-de2b79fe3b61 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:26:46.677198 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71350a6-bc7f-4586-a2aa-de2b79fe3b61 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:26:46.706494 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:26:46.744543 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71350a6-bc7f-4586-a2aa-de2b79fe3b61 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:26:46.745965 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71350a6-bc7f-4586-a2aa-de2b79fe3b61 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:26:46.846374 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 603/1214] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Aug 10 08:26:46 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 08:26:46.852312 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a9f9f9-b442-4502-9bb4-2e77c8254833 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:26:46.872076 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:26:46.952588 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a9f9f9-b442-4502-9bb4-2e77c8254833 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:26:46.986379 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3809475-b115-4311-a631-a90e3308c323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:26:46.987001 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3809475-b115-4311-a631-a90e3308c323 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:26:46.987276 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3809475-b115-4311-a631-a90e3308c323 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:26:46.987547 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3809475-b115-4311-a631-a90e3308c323 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:26:46.997907 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3809475-b115-4311-a631-a90e3308c323 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:26:46.999280 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3809475-b115-4311-a631-a90e3308c323 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:26:47.014411 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-e3809475-b115-4311-a631-a90e3308c323 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:26:47.014411 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:26:47.014411 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:26:47.014411 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:26:47.014411 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:26:47.014411 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:26:47.014411 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:26:47.014411 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:26:47.014411 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:26:47.014411 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:26:47.014411 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:26:47.014411 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:26:47.014411 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 08:26:47.014411 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 08:26:47.014411 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 08:26:47.014411 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 08:26:47.015600 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:26:47.015600 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:26:47.015600 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:26:47.015600 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:26:47.015600 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:26:47.015600 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:26:47.015600 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:26:47.015600 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:26:47.015600 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:26:47.015600 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:26:47.015600 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:26:47.015600 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:26:47.015600 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:26:47.015600 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:26:47.015600 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:26:47.015600 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:26:47.015600 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 08:26:47.016712 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 08:26:47.016712 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:26:47.016712 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:26:47.016712 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 612/1215] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:26:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:26:47.020939 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71350a6-bc7f-4586-a2aa-de2b79fe3b61 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:26:47.081863 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:26:47.119298 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71350a6-bc7f-4586-a2aa-de2b79fe3b61 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:26:47.153019 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2c9bed-7eac-4b62-b16b-da1769c36d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:26:47.154276 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2c9bed-7eac-4b62-b16b-da1769c36d7d None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:26:47.154527 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2c9bed-7eac-4b62-b16b-da1769c36d7d None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:26:47.154793 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2c9bed-7eac-4b62-b16b-da1769c36d7d None admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:26:47.155422 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2c9bed-7eac-4b62-b16b-da1769c36d7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:26:47.155720 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2c9bed-7eac-4b62-b16b-da1769c36d7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:26:47.157106 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2c9bed-7eac-4b62-b16b-da1769c36d7d None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:26:47.169100 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 604/1216] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Aug 10 08:26:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:26:47.174604 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3809475-b115-4311-a631-a90e3308c323 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:26:47.242882 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:26:47.272539 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3809475-b115-4311-a631-a90e3308c323 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:26:47.304828 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c52873-5162-426f-99f2-bf40d311e9bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:26:47.305476 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c52873-5162-426f-99f2-bf40d311e9bb None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:26:47.305758 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c52873-5162-426f-99f2-bf40d311e9bb None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:26:47.306031 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c52873-5162-426f-99f2-bf40d311e9bb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:26:47.312518 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c52873-5162-426f-99f2-bf40d311e9bb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:26:47.313988 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c52873-5162-426f-99f2-bf40d311e9bb None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:26:47.324861 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-46c52873-5162-426f-99f2-bf40d311e9bb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 08:26:47.324861 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:26:47.324861 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:26:47.324861 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:26:47.324861 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:26:47.324861 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:26:47.324861 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:26:47.324861 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:26:47.324861 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:26:47.324861 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:26:47.324861 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:26:47.324861 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:26:47.324861 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 08:26:47.324861 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 08:26:47.324861 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 08:26:47.326607 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 08:26:47.326607 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:26:47.326607 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:26:47.326607 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:26:47.326607 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:26:47.326607 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:26:47.326607 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:26:47.326607 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:26:47.326607 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:26:47.326607 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:26:47.326607 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:26:47.326607 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:26:47.326607 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:26:47.326607 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:26:47.326607 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:26:47.326607 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:26:47.326607 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:26:47.327762 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 08:26:47.327762 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 08:26:47.327762 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 08:26:47.327762 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 08:26:47.327762 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 08:26:47.327762 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 08:26:47.327762 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 08:26:47.327762 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:26:47.327762 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 613/1217] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Aug 10 08:26:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:26:47.332508 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2c9bed-7eac-4b62-b16b-da1769c36d7d None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:26:47.431765 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2c9bed-7eac-4b62-b16b-da1769c36d7d None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:26:47.454700 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:26:47.465539 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f6ab50-8306-42d4-8fe6-c7bbaae29c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:26:47.466299 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f6ab50-8306-42d4-8fe6-c7bbaae29c11 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:26:47.466419 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f6ab50-8306-42d4-8fe6-c7bbaae29c11 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:26:47.466684 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f6ab50-8306-42d4-8fe6-c7bbaae29c11 None admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:26:47.467401 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f6ab50-8306-42d4-8fe6-c7bbaae29c11 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:26:47.467697 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f6ab50-8306-42d4-8fe6-c7bbaae29c11 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:26:47.469136 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f6ab50-8306-42d4-8fe6-c7bbaae29c11 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:26:47.480387 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 605/1218] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Aug 10 08:26:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:26:47.487131 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c52873-5162-426f-99f2-bf40d311e9bb None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:26:47.587632 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c52873-5162-426f-99f2-bf40d311e9bb None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:26:47.614856 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:26:47.620701 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d370dd99-276d-46bd-bf59-c5e2c6522bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:26:47.621326 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d370dd99-276d-46bd-bf59-c5e2c6522bc2 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:26:47.621607 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d370dd99-276d-46bd-bf59-c5e2c6522bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:26:47.621878 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d370dd99-276d-46bd-bf59-c5e2c6522bc2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:26:47.623135 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d370dd99-276d-46bd-bf59-c5e2c6522bc2 None admin] RBAC: Adding query filter params (role.id=73fdd60e2f804ee69b76fa9145a472ff, scope.project.id=b49e381419e940278bcb6111899589b5, user.id=2912307c56594125b157f1cfdf7c96a9) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:26:47.623435 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d370dd99-276d-46bd-bf59-c5e2c6522bc2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:26:47.624852 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d370dd99-276d-46bd-bf59-c5e2c6522bc2 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:26:47.630071 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 614/1219] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Aug 10 08:26:47 2022] GET /identity/v3/role_assignments?user.id=2912307c56594125b157f1cfdf7c96a9&scope.project.id=b49e381419e940278bcb6111899589b5&role.id=73fdd60e2f804ee69b76fa9145a472ff => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:26:49.636417 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d72ce0-c201-469a-aa77-1a9e85a0ea3a None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:26:49.637031 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d72ce0-c201-469a-aa77-1a9e85a0ea3a None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:26:49.637031 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d72ce0-c201-469a-aa77-1a9e85a0ea3a None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:26:49.639076 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 606/1220] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:26:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:26:49.644312 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763a6aef-d06a-44f4-b352-92e593104f15 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:26:49.644713 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763a6aef-d06a-44f4-b352-92e593104f15 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:26:49.644859 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763a6aef-d06a-44f4-b352-92e593104f15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:26:49.662837 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-763a6aef-d06a-44f4-b352-92e593104f15 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:26:49.664026 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-763a6aef-d06a-44f4-b352-92e593104f15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:26:49.740193 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 615/1221] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:26:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 08:26:49.764095 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d72ce0-c201-469a-aa77-1a9e85a0ea3a None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:26:49.864195 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d72ce0-c201-469a-aa77-1a9e85a0ea3a None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:26:49.900978 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70be04b-69db-4385-901d-bfdb6b1e50e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:26:49.902950 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70be04b-69db-4385-901d-bfdb6b1e50e2 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:26:49.902950 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70be04b-69db-4385-901d-bfdb6b1e50e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:26:49.902950 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70be04b-69db-4385-901d-bfdb6b1e50e2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:26:49.917406 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70be04b-69db-4385-901d-bfdb6b1e50e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 08:26:49.923825 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-a70be04b-69db-4385-901d-bfdb6b1e50e2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 08:26:49.923825 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:26:49.923825 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:26:49.923825 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:26:49.923825 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:26:49.923825 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:26:49.923825 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:26:49.923825 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:26:49.923825 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:26:49.923825 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:26:49.923825 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:26:49.923825 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:26:49.923825 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 08:26:49.923825 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 08:26:49.923825 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 08:26:49.923825 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 08:26:49.927410 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 08:26:49.927410 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 08:26:49.927410 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 08:26:49.927410 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 08:26:49.927410 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:26:49.927410 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:26:49.927410 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:26:49.927410 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:26:49.927410 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:26:49.927410 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:26:49.927410 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:26:49.927410 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:26:49.927410 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:26:49.927410 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:26:49.927410 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:26:49.927410 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:26:49.928494 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:26:49.928494 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:26:49.928494 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:26:49.928494 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:26:49.928494 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 08:26:49.928494 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 08:26:49.928494 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 08:26:49.928494 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 08:26:49.928494 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 08:26:49.928494 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 08:26:49.928494 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 08:26:49.928494 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:26:49.928494 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 607/1222] 10.222.0.57 () {56 vars in 1102 bytes} [Wed Aug 10 08:26:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:26:49.931007 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763a6aef-d06a-44f4-b352-92e593104f15 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:26:50.027454 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763a6aef-d06a-44f4-b352-92e593104f15 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:26:50.050862 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:26:50.062450 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8807a06a-0a24-43c4-8eb1-d720f3e82183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:26:50.063058 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8807a06a-0a24-43c4-8eb1-d720f3e82183 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:26:50.063334 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8807a06a-0a24-43c4-8eb1-d720f3e82183 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:26:50.063602 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8807a06a-0a24-43c4-8eb1-d720f3e82183 None admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:26:50.064258 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8807a06a-0a24-43c4-8eb1-d720f3e82183 None admin] RBAC: Adding query filter params (name=service) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:26:50.064557 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8807a06a-0a24-43c4-8eb1-d720f3e82183 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:26:50.065944 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8807a06a-0a24-43c4-8eb1-d720f3e82183 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:26:50.074069 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 616/1223] 10.222.0.57 () {56 vars in 1103 bytes} [Wed Aug 10 08:26:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:26:50.080568 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70be04b-69db-4385-901d-bfdb6b1e50e2 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:26:50.129265 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:26:50.184112 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70be04b-69db-4385-901d-bfdb6b1e50e2 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:26:50.218525 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e38ac20-62d1-409f-bad7-b8d1818cfa9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:26:50.219150 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e38ac20-62d1-409f-bad7-b8d1818cfa9e None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:26:50.219444 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e38ac20-62d1-409f-bad7-b8d1818cfa9e None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:26:50.219698 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e38ac20-62d1-409f-bad7-b8d1818cfa9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:26:50.229995 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e38ac20-62d1-409f-bad7-b8d1818cfa9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:26:50.231433 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e38ac20-62d1-409f-bad7-b8d1818cfa9e None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:26:50.245902 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-6e38ac20-62d1-409f-bad7-b8d1818cfa9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:26:50.245902 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:26:50.245902 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:26:50.245902 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:26:50.245902 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:26:50.245902 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:26:50.245902 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:26:50.245902 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:26:50.245902 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:26:50.245902 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:26:50.245902 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:26:50.245902 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:26:50.245902 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 08:26:50.245902 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 08:26:50.245902 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 08:26:50.245902 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 08:26:50.247062 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:26:50.247062 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:26:50.247062 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:26:50.247062 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:26:50.247062 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:26:50.247062 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:26:50.247062 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:26:50.247062 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:26:50.247062 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:26:50.247062 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:26:50.247062 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:26:50.247062 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:26:50.247062 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:26:50.247062 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:26:50.247062 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:26:50.247062 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:26:50.247062 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 08:26:50.248248 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 08:26:50.248248 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:26:50.248248 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:26:50.248248 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 608/1224] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:26:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:26:50.253406 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8807a06a-0a24-43c4-8eb1-d720f3e82183 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:26:50.351559 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8807a06a-0a24-43c4-8eb1-d720f3e82183 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:26:50.385077 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c480883a-7276-400d-9511-a995b97974b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:26:50.385680 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c480883a-7276-400d-9511-a995b97974b0 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:26:50.385955 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c480883a-7276-400d-9511-a995b97974b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:26:50.386222 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c480883a-7276-400d-9511-a995b97974b0 None admin] PATH_INFO: `/v3/domains` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:26:50.386862 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c480883a-7276-400d-9511-a995b97974b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:26:50.387244 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c480883a-7276-400d-9511-a995b97974b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:26:50.388613 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c480883a-7276-400d-9511-a995b97974b0 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:26:50.400979 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 617/1225] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Aug 10 08:26:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:26:50.407616 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e38ac20-62d1-409f-bad7-b8d1818cfa9e None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:26:50.501297 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:26:50.507992 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e38ac20-62d1-409f-bad7-b8d1818cfa9e None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:26:50.540839 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a9159b-00cf-4e20-a1b0-cd94f5905dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:26:50.541489 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a9159b-00cf-4e20-a1b0-cd94f5905dba None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:26:50.541769 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a9159b-00cf-4e20-a1b0-cd94f5905dba None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:26:50.542043 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a9159b-00cf-4e20-a1b0-cd94f5905dba None admin] PATH_INFO: `/v3/users/placement` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:26:50.550020 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a9159b-00cf-4e20-a1b0-cd94f5905dba None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:26:50.551460 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a9159b-00cf-4e20-a1b0-cd94f5905dba None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:26:50.562180 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-87a9159b-00cf-4e20-a1b0-cd94f5905dba None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 10 08:26:50.562180 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:26:50.562180 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:26:50.562180 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:26:50.562180 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:26:50.562180 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:26:50.562180 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:26:50.562180 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:26:50.562180 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:26:50.562180 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:26:50.562180 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:26:50.562180 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:26:50.562180 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 08:26:50.562180 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 08:26:50.562180 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 08:26:50.562180 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 08:26:50.568399 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:26:50.568399 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:26:50.568399 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 08:26:50.568399 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 08:26:50.568399 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 08:26:50.568399 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 08:26:50.568399 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:26:50.568399 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:26:50.568399 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:26:50.568399 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:26:50.568399 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:26:50.568399 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:26:50.568399 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:26:50.568399 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:26:50.568399 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:26:50.568399 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:26:50.568399 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:26:50.568399 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:26:50.569632 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:26:50.569632 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:26:50.569632 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 08:26:50.569632 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 08:26:50.569632 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 08:26:50.569632 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 08:26:50.569632 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 08:26:50.569632 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 08:26:50.569632 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 10 08:26:50.569632 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:26:50.569632 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 609/1226] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:26:50 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:26:50.570415 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c480883a-7276-400d-9511-a995b97974b0 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:26:50.616774 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:26:50.667471 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c480883a-7276-400d-9511-a995b97974b0 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:26:50.705554 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a541c75-1f84-46bd-b97a-d47322d3cbdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:26:50.706135 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a541c75-1f84-46bd-b97a-d47322d3cbdd None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:26:50.706412 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a541c75-1f84-46bd-b97a-d47322d3cbdd None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:26:50.706687 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a541c75-1f84-46bd-b97a-d47322d3cbdd None admin] PATH_INFO: `/v3/users` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:26:50.707443 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a541c75-1f84-46bd-b97a-d47322d3cbdd None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:26:50.707760 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a541c75-1f84-46bd-b97a-d47322d3cbdd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:26:50.709566 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a541c75-1f84-46bd-b97a-d47322d3cbdd None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:26:50.727849 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 618/1227] 10.222.0.57 () {56 vars in 1143 bytes} [Wed Aug 10 08:26:50 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:26:50.733554 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a9159b-00cf-4e20-a1b0-cd94f5905dba None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:26:50.831476 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a9159b-00cf-4e20-a1b0-cd94f5905dba None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:26:50.864446 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae13b150-d254-4403-9994-bc8ff6148a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:26:50.865113 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae13b150-d254-4403-9994-bc8ff6148a35 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:26:50.865392 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae13b150-d254-4403-9994-bc8ff6148a35 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:26:50.865659 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae13b150-d254-4403-9994-bc8ff6148a35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:26:50.876226 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae13b150-d254-4403-9994-bc8ff6148a35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:26:50.877705 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae13b150-d254-4403-9994-bc8ff6148a35 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:26:50.891832 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-ae13b150-d254-4403-9994-bc8ff6148a35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:26:50.891832 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:26:50.891832 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:26:50.891832 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:26:50.891832 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:26:50.891832 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:26:50.891832 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:26:50.891832 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:26:50.891832 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:26:50.891832 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:26:50.891832 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:26:50.891832 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:26:50.891832 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 08:26:50.891832 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 08:26:50.891832 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 08:26:50.891832 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 08:26:50.893137 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:26:50.893137 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:26:50.893137 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:26:50.893137 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:26:50.893137 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:26:50.893137 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:26:50.893137 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:26:50.893137 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:26:50.893137 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:26:50.893137 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:26:50.893137 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:26:50.893137 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:26:50.893137 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:26:50.893137 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:26:50.893137 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:26:50.893137 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:26:50.893137 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 08:26:50.894489 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 08:26:50.894489 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:26:50.894489 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:26:50.894489 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 610/1228] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:26:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:26:50.898725 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a541c75-1f84-46bd-b97a-d47322d3cbdd None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:26:50.998194 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a541c75-1f84-46bd-b97a-d47322d3cbdd None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:26:51.000587 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:26:51.032351 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84bc194-f000-41ad-be3d-e96cac1fc653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:26:51.032973 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84bc194-f000-41ad-be3d-e96cac1fc653 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:26:51.033254 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84bc194-f000-41ad-be3d-e96cac1fc653 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:26:51.033521 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84bc194-f000-41ad-be3d-e96cac1fc653 None admin] PATH_INFO: `/v3/domains` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:26:51.034152 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84bc194-f000-41ad-be3d-e96cac1fc653 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:26:51.034451 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84bc194-f000-41ad-be3d-e96cac1fc653 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:26:51.035803 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84bc194-f000-41ad-be3d-e96cac1fc653 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:26:51.046476 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 619/1229] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Aug 10 08:26:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:26:51.052167 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae13b150-d254-4403-9994-bc8ff6148a35 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:26:51.062297 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:26:51.150472 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae13b150-d254-4403-9994-bc8ff6148a35 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:26:51.183515 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba02060-2750-4f30-9817-415e70391864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:26:51.184133 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba02060-2750-4f30-9817-415e70391864 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:26:51.184406 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba02060-2750-4f30-9817-415e70391864 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:26:51.184669 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba02060-2750-4f30-9817-415e70391864 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:26:51.192125 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba02060-2750-4f30-9817-415e70391864 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:26:51.193579 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba02060-2750-4f30-9817-415e70391864 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:26:51.208100 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-cba02060-2750-4f30-9817-415e70391864 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 08:26:51.208100 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:26:51.208100 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:26:51.208100 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:26:51.208100 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:26:51.208100 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:26:51.208100 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:26:51.208100 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:26:51.208100 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:26:51.208100 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:26:51.208100 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:26:51.208100 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:26:51.208100 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 08:26:51.208100 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 08:26:51.208100 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 08:26:51.211284 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 08:26:51.211284 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:26:51.211284 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:26:51.211284 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:26:51.211284 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:26:51.211284 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:26:51.211284 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:26:51.211284 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:26:51.211284 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:26:51.211284 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:26:51.211284 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:26:51.211284 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:26:51.211284 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:26:51.211284 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:26:51.211284 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:26:51.211284 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:26:51.211284 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:26:51.212420 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 08:26:51.212420 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 08:26:51.212420 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 08:26:51.212420 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 08:26:51.212420 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 08:26:51.212420 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 08:26:51.212420 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 08:26:51.212420 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:26:51.212420 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 611/1230] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Aug 10 08:26:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:26:51.215340 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84bc194-f000-41ad-be3d-e96cac1fc653 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:26:51.316973 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84bc194-f000-41ad-be3d-e96cac1fc653 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:26:51.350780 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f45139-0809-45cb-abe7-306419567e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:26:51.351385 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f45139-0809-45cb-abe7-306419567e65 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:26:51.351660 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f45139-0809-45cb-abe7-306419567e65 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:26:51.351938 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f45139-0809-45cb-abe7-306419567e65 None admin] PATH_INFO: `/v3/projects` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:26:51.352691 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f45139-0809-45cb-abe7-306419567e65 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:26:51.353021 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f45139-0809-45cb-abe7-306419567e65 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:26:51.357457 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f45139-0809-45cb-abe7-306419567e65 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:26:51.366531 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 620/1231] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Aug 10 08:26:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:26:51.372793 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba02060-2750-4f30-9817-415e70391864 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:26:51.437695 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:26:51.471165 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba02060-2750-4f30-9817-415e70391864 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:26:51.505157 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5ca6fe-8a80-4e99-9e31-efa164e72dfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:26:51.505800 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5ca6fe-8a80-4e99-9e31-efa164e72dfb None admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:26:51.506076 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5ca6fe-8a80-4e99-9e31-efa164e72dfb None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:26:51.506346 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5ca6fe-8a80-4e99-9e31-efa164e72dfb None admin] PATH_INFO: `/v3/projects/b49e381419e940278bcb6111899589b5/users/2912307c56594125b157f1cfdf7c96a9/roles/73fdd60e2f804ee69b76fa9145a472ff` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:26:51.520295 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5ca6fe-8a80-4e99-9e31-efa164e72dfb None admin] RBAC: Authorizing `identity:create_grant(project_id=b49e381419e940278bcb6111899589b5, user_id=2912307c56594125b157f1cfdf7c96a9, role_id=73fdd60e2f804ee69b76fa9145a472ff)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:26:51.521746 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5ca6fe-8a80-4e99-9e31-efa164e72dfb None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:26:51.587813 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 612/1232] 10.222.0.57 () {58 vars in 1439 bytes} [Wed Aug 10 08:26:51 2022] PUT /identity/v3/projects/b49e381419e940278bcb6111899589b5/users/2912307c56594125b157f1cfdf7c96a9/roles/73fdd60e2f804ee69b76fa9145a472ff => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:26:53.552797 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cc72d0-519b-4f70-9cdd-e05f806b4438 None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:26:53.553365 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cc72d0-519b-4f70-9cdd-e05f806b4438 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:26:53.553475 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cc72d0-519b-4f70-9cdd-e05f806b4438 None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:26:53.554799 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 621/1233] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:26:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:26:53.560144 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d46c0f-b3eb-4603-bd8d-0265d980fd8b None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:26:53.560575 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d46c0f-b3eb-4603-bd8d-0265d980fd8b None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:26:53.560698 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d46c0f-b3eb-4603-bd8d-0265d980fd8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:26:53.579404 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-55d46c0f-b3eb-4603-bd8d-0265d980fd8b None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:26:53.580569 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-55d46c0f-b3eb-4603-bd8d-0265d980fd8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:26:53.646596 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 613/1234] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:26:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 08:26:53.669703 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cc72d0-519b-4f70-9cdd-e05f806b4438 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:26:53.695885 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:26:53.770940 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cc72d0-519b-4f70-9cdd-e05f806b4438 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:26:53.803601 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee8738e-d158-4bb5-8a37-c1d04916eccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:26:53.804263 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee8738e-d158-4bb5-8a37-c1d04916eccd None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:26:53.804538 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee8738e-d158-4bb5-8a37-c1d04916eccd None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:26:53.804807 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee8738e-d158-4bb5-8a37-c1d04916eccd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:26:53.815275 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee8738e-d158-4bb5-8a37-c1d04916eccd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 08:26:53.819926 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-4ee8738e-d158-4bb5-8a37-c1d04916eccd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 08:26:53.819926 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:26:53.819926 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:26:53.819926 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:26:53.819926 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:26:53.819926 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:26:53.819926 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:26:53.819926 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:26:53.819926 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:26:53.819926 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:26:53.819926 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:26:53.819926 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:26:53.819926 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 08:26:53.819926 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 08:26:53.819926 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 08:26:53.819926 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 08:26:53.822757 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 08:26:53.822757 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 08:26:53.822757 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 08:26:53.822757 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 08:26:53.822757 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:26:53.822757 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:26:53.822757 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:26:53.822757 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:26:53.822757 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:26:53.822757 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:26:53.822757 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:26:53.822757 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:26:53.822757 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:26:53.822757 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:26:53.822757 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:26:53.822757 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:26:53.823883 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:26:53.823883 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:26:53.823883 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:26:53.823883 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:26:53.823883 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 08:26:53.823883 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 08:26:53.823883 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 08:26:53.823883 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 08:26:53.823883 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 08:26:53.823883 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 08:26:53.823883 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 08:26:53.823883 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:26:53.823883 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 622/1235] 10.222.0.57 () {56 vars in 1102 bytes} [Wed Aug 10 08:26:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:26:53.827195 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d46c0f-b3eb-4603-bd8d-0265d980fd8b None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:26:53.873418 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:26:53.926183 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d46c0f-b3eb-4603-bd8d-0265d980fd8b None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:26:53.959418 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79df05bc-bfb2-4d6e-92ca-ad7ff70b5f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:26:53.960010 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79df05bc-bfb2-4d6e-92ca-ad7ff70b5f92 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:26:53.960284 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79df05bc-bfb2-4d6e-92ca-ad7ff70b5f92 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:26:53.960549 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79df05bc-bfb2-4d6e-92ca-ad7ff70b5f92 None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:26:53.961231 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79df05bc-bfb2-4d6e-92ca-ad7ff70b5f92 None admin] RBAC: Adding query filter params (name=service) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:26:53.961527 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79df05bc-bfb2-4d6e-92ca-ad7ff70b5f92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:26:53.962873 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79df05bc-bfb2-4d6e-92ca-ad7ff70b5f92 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:26:53.970431 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 614/1236] 10.222.0.57 () {56 vars in 1103 bytes} [Wed Aug 10 08:26:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:26:53.976352 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee8738e-d158-4bb5-8a37-c1d04916eccd None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:26:54.063954 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:26:54.078720 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee8738e-d158-4bb5-8a37-c1d04916eccd None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:26:54.112005 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ef4484-53d7-471e-bd87-a831624a487c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:26:54.112789 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ef4484-53d7-471e-bd87-a831624a487c None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:26:54.113027 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ef4484-53d7-471e-bd87-a831624a487c None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:26:54.113299 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ef4484-53d7-471e-bd87-a831624a487c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:26:54.123285 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ef4484-53d7-471e-bd87-a831624a487c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:26:54.124844 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ef4484-53d7-471e-bd87-a831624a487c None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:26:54.138777 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-83ef4484-53d7-471e-bd87-a831624a487c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:26:54.138777 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:26:54.138777 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:26:54.138777 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:26:54.138777 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:26:54.138777 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:26:54.138777 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:26:54.138777 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:26:54.138777 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:26:54.138777 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:26:54.138777 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:26:54.138777 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:26:54.138777 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 08:26:54.138777 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 08:26:54.138777 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 08:26:54.138777 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 08:26:54.139941 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:26:54.139941 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:26:54.139941 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:26:54.139941 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:26:54.139941 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:26:54.139941 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:26:54.139941 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:26:54.139941 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:26:54.139941 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:26:54.139941 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:26:54.139941 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:26:54.139941 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:26:54.139941 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:26:54.139941 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:26:54.139941 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:26:54.139941 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:26:54.139941 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 08:26:54.141137 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 08:26:54.141137 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:26:54.141137 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:26:54.141137 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 623/1237] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:26:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:26:54.146014 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79df05bc-bfb2-4d6e-92ca-ad7ff70b5f92 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:26:54.245417 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79df05bc-bfb2-4d6e-92ca-ad7ff70b5f92 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:26:54.254346 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:26:54.278171 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d063783d-fe0e-40a3-8db9-9efb1cd92add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:26:54.278768 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d063783d-fe0e-40a3-8db9-9efb1cd92add None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:26:54.279042 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d063783d-fe0e-40a3-8db9-9efb1cd92add None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:26:54.279309 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d063783d-fe0e-40a3-8db9-9efb1cd92add None admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:26:54.280447 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d063783d-fe0e-40a3-8db9-9efb1cd92add None admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:26:54.281226 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d063783d-fe0e-40a3-8db9-9efb1cd92add None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:26:54.282582 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d063783d-fe0e-40a3-8db9-9efb1cd92add None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:26:54.293690 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 615/1238] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Aug 10 08:26:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:26:54.300615 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ef4484-53d7-471e-bd87-a831624a487c None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:26:54.400382 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ef4484-53d7-471e-bd87-a831624a487c None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:26:54.434435 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e297d6d0-ed7e-4ffc-9787-b8736cd87b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:26:54.435062 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e297d6d0-ed7e-4ffc-9787-b8736cd87b7c None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:26:54.435344 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e297d6d0-ed7e-4ffc-9787-b8736cd87b7c None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:26:54.435617 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e297d6d0-ed7e-4ffc-9787-b8736cd87b7c None admin] PATH_INFO: `/v3/users/placement` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:26:54.438384 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:26:54.444024 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e297d6d0-ed7e-4ffc-9787-b8736cd87b7c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:26:54.445606 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e297d6d0-ed7e-4ffc-9787-b8736cd87b7c None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:26:54.457108 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-e297d6d0-ed7e-4ffc-9787-b8736cd87b7c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 10 08:26:54.457108 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:26:54.457108 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:26:54.457108 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:26:54.457108 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:26:54.457108 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:26:54.457108 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:26:54.457108 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:26:54.457108 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:26:54.457108 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:26:54.457108 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:26:54.457108 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:26:54.457108 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 08:26:54.457108 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 08:26:54.457108 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 08:26:54.457108 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 08:26:54.458125 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:26:54.458125 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:26:54.458125 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 08:26:54.458125 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 08:26:54.458125 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 08:26:54.458125 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 08:26:54.458125 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:26:54.458125 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:26:54.458125 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:26:54.458125 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:26:54.458125 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:26:54.458125 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:26:54.458125 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:26:54.458125 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:26:54.458125 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:26:54.458125 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:26:54.458125 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:26:54.458125 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:26:54.459107 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:26:54.459107 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:26:54.459107 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 08:26:54.459107 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 08:26:54.459107 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 08:26:54.459107 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 08:26:54.459107 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 08:26:54.459107 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 08:26:54.459107 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 10 08:26:54.459107 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:26:54.459107 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 624/1239] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:26:54 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:26:54.464596 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d063783d-fe0e-40a3-8db9-9efb1cd92add None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:26:54.564553 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d063783d-fe0e-40a3-8db9-9efb1cd92add None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:26:54.599857 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801eb0e2-d127-4e4f-9e0a-4a8d63c85062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:26:54.600450 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801eb0e2-d127-4e4f-9e0a-4a8d63c85062 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:26:54.600727 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801eb0e2-d127-4e4f-9e0a-4a8d63c85062 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:26:54.601024 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801eb0e2-d127-4e4f-9e0a-4a8d63c85062 None admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:26:54.601751 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801eb0e2-d127-4e4f-9e0a-4a8d63c85062 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:26:54.602054 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801eb0e2-d127-4e4f-9e0a-4a8d63c85062 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:26:54.603472 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801eb0e2-d127-4e4f-9e0a-4a8d63c85062 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:26:54.620566 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 616/1240] 10.222.0.57 () {56 vars in 1143 bytes} [Wed Aug 10 08:26:54 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:26:54.626915 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e297d6d0-ed7e-4ffc-9787-b8736cd87b7c None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:26:54.724669 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e297d6d0-ed7e-4ffc-9787-b8736cd87b7c None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:26:54.761873 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93af346-3752-4109-b237-2d0b0f0aec82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:26:54.762564 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93af346-3752-4109-b237-2d0b0f0aec82 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:26:54.762862 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93af346-3752-4109-b237-2d0b0f0aec82 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:26:54.763265 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93af346-3752-4109-b237-2d0b0f0aec82 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:26:54.826023 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:26:54.833380 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93af346-3752-4109-b237-2d0b0f0aec82 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:26:54.834873 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93af346-3752-4109-b237-2d0b0f0aec82 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:26:54.938939 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 625/1241] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Aug 10 08:26:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 08:26:54.945003 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801eb0e2-d127-4e4f-9e0a-4a8d63c85062 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:26:54.973366 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:26:55.047841 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801eb0e2-d127-4e4f-9e0a-4a8d63c85062 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:26:55.084688 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c157a711-2112-4919-a64d-242402bd84a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:26:55.085325 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c157a711-2112-4919-a64d-242402bd84a2 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:26:55.085670 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c157a711-2112-4919-a64d-242402bd84a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:26:55.085947 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c157a711-2112-4919-a64d-242402bd84a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:26:55.096059 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c157a711-2112-4919-a64d-242402bd84a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:26:55.097538 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c157a711-2112-4919-a64d-242402bd84a2 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:26:55.111860 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-c157a711-2112-4919-a64d-242402bd84a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:26:55.111860 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:26:55.111860 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:26:55.111860 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:26:55.111860 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:26:55.111860 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:26:55.111860 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:26:55.111860 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:26:55.111860 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:26:55.111860 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:26:55.111860 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:26:55.111860 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:26:55.111860 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 08:26:55.111860 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 08:26:55.111860 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 08:26:55.111860 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 08:26:55.113109 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:26:55.113109 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:26:55.113109 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:26:55.113109 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:26:55.113109 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:26:55.113109 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:26:55.113109 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:26:55.113109 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:26:55.113109 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:26:55.113109 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:26:55.113109 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:26:55.113109 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:26:55.113109 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:26:55.113109 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:26:55.113109 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:26:55.113109 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:26:55.113109 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 08:26:55.114096 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 08:26:55.114096 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:26:55.114096 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:26:55.114096 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 617/1242] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:26:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 08:26:55.119116 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93af346-3752-4109-b237-2d0b0f0aec82 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:26:55.214154 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:26:55.219763 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93af346-3752-4109-b237-2d0b0f0aec82 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:26:55.253120 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10570aa6-c2b5-45bb-8c1f-08b92a259a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:26:55.253717 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10570aa6-c2b5-45bb-8c1f-08b92a259a2c None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:26:55.253996 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10570aa6-c2b5-45bb-8c1f-08b92a259a2c None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:26:55.254270 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10570aa6-c2b5-45bb-8c1f-08b92a259a2c None admin] PATH_INFO: `/v3/domains` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:26:55.254898 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10570aa6-c2b5-45bb-8c1f-08b92a259a2c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:26:55.255198 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10570aa6-c2b5-45bb-8c1f-08b92a259a2c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:26:55.256589 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10570aa6-c2b5-45bb-8c1f-08b92a259a2c None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:26:55.267745 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 626/1243] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Aug 10 08:26:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:26:55.273599 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c157a711-2112-4919-a64d-242402bd84a2 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:26:55.351781 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:26:55.373499 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c157a711-2112-4919-a64d-242402bd84a2 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:26:55.407417 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71a4e8b-0cbd-4838-80a3-08b49fe365da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:26:55.408036 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71a4e8b-0cbd-4838-80a3-08b49fe365da None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:26:55.408310 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71a4e8b-0cbd-4838-80a3-08b49fe365da None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:26:55.408574 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71a4e8b-0cbd-4838-80a3-08b49fe365da None admin] PATH_INFO: `/v3/projects/service` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:26:55.415360 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71a4e8b-0cbd-4838-80a3-08b49fe365da None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:26:55.416832 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71a4e8b-0cbd-4838-80a3-08b49fe365da None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:26:55.428476 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-e71a4e8b-0cbd-4838-80a3-08b49fe365da None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 08:26:55.428476 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:26:55.428476 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:26:55.428476 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:26:55.428476 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:26:55.428476 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:26:55.428476 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:26:55.428476 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:26:55.428476 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:26:55.428476 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:26:55.428476 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:26:55.428476 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:26:55.428476 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 08:26:55.428476 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 08:26:55.428476 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 08:26:55.431264 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 08:26:55.431264 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:26:55.431264 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:26:55.431264 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:26:55.431264 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:26:55.431264 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:26:55.431264 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:26:55.431264 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:26:55.431264 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:26:55.431264 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:26:55.431264 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:26:55.431264 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:26:55.431264 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:26:55.431264 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:26:55.431264 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:26:55.431264 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:26:55.431264 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:26:55.432393 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 08:26:55.432393 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 08:26:55.432393 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 08:26:55.432393 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 08:26:55.432393 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 08:26:55.432393 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 08:26:55.432393 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 08:26:55.432393 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:26:55.432393 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 618/1244] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Aug 10 08:26:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:26:55.435655 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10570aa6-c2b5-45bb-8c1f-08b92a259a2c None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:26:55.540298 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10570aa6-c2b5-45bb-8c1f-08b92a259a2c None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:26:55.573741 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcae801-013e-40a8-a58a-b5b478bd0894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:26:55.574347 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcae801-013e-40a8-a58a-b5b478bd0894 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:26:55.574628 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcae801-013e-40a8-a58a-b5b478bd0894 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:26:55.574898 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcae801-013e-40a8-a58a-b5b478bd0894 None admin] PATH_INFO: `/v3/projects` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:26:55.575579 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcae801-013e-40a8-a58a-b5b478bd0894 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:26:55.575914 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcae801-013e-40a8-a58a-b5b478bd0894 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:26:55.577377 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcae801-013e-40a8-a58a-b5b478bd0894 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:26:55.589833 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 627/1245] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Aug 10 08:26:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:26:55.596618 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71a4e8b-0cbd-4838-80a3-08b49fe365da None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:26:55.694683 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71a4e8b-0cbd-4838-80a3-08b49fe365da None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:26:55.728368 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc1e97e-f6b3-4306-bff1-b6cb7ae6ef96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:26:55.728986 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc1e97e-f6b3-4306-bff1-b6cb7ae6ef96 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:26:55.729268 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc1e97e-f6b3-4306-bff1-b6cb7ae6ef96 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:26:55.729538 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc1e97e-f6b3-4306-bff1-b6cb7ae6ef96 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:26:55.730258 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc1e97e-f6b3-4306-bff1-b6cb7ae6ef96 None admin] RBAC: Adding query filter params (role.id=73fdd60e2f804ee69b76fa9145a472ff, scope.project.id=b49e381419e940278bcb6111899589b5, user.id=2912307c56594125b157f1cfdf7c96a9) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:26:55.730558 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc1e97e-f6b3-4306-bff1-b6cb7ae6ef96 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:26:55.731970 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc1e97e-f6b3-4306-bff1-b6cb7ae6ef96 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:26:55.737817 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 619/1246] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Aug 10 08:26:55 2022] GET /identity/v3/role_assignments?user.id=2912307c56594125b157f1cfdf7c96a9&scope.project.id=b49e381419e940278bcb6111899589b5&role.id=73fdd60e2f804ee69b76fa9145a472ff => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:26:57.813884 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5446f6-c740-49f8-9732-f6c31d371364 None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:26:57.814414 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5446f6-c740-49f8-9732-f6c31d371364 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:26:57.814528 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5446f6-c740-49f8-9732-f6c31d371364 None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:26:57.815791 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 628/1247] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:26:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:26:57.821267 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a171e7ea-53aa-4f8a-9c24-8e2a04361593 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:26:57.821719 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a171e7ea-53aa-4f8a-9c24-8e2a04361593 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:26:57.821821 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a171e7ea-53aa-4f8a-9c24-8e2a04361593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:26:57.828053 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:26:57.839920 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-a171e7ea-53aa-4f8a-9c24-8e2a04361593 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:26:57.841132 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-a171e7ea-53aa-4f8a-9c24-8e2a04361593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:26:57.909499 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 620/1248] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:26:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 08:26:57.933272 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5446f6-c740-49f8-9732-f6c31d371364 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:26:57.951056 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:26:58.036636 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5446f6-c740-49f8-9732-f6c31d371364 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:26:58.071218 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf69cc0-6766-4237-aee6-f37b5fac6997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:26:58.071775 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf69cc0-6766-4237-aee6-f37b5fac6997 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:26:58.072052 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf69cc0-6766-4237-aee6-f37b5fac6997 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:26:58.072322 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf69cc0-6766-4237-aee6-f37b5fac6997 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:26:58.081990 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf69cc0-6766-4237-aee6-f37b5fac6997 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 08:26:58.086548 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-abf69cc0-6766-4237-aee6-f37b5fac6997 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 08:26:58.086548 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:26:58.086548 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:26:58.086548 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:26:58.086548 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:26:58.086548 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:26:58.086548 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:26:58.086548 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:26:58.086548 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:26:58.086548 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:26:58.086548 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:26:58.086548 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:26:58.086548 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 08:26:58.086548 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 08:26:58.086548 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 08:26:58.086548 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 08:26:58.087681 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 08:26:58.087681 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 08:26:58.087681 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 08:26:58.087681 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 08:26:58.087681 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:26:58.087681 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:26:58.087681 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:26:58.087681 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:26:58.087681 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:26:58.087681 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:26:58.087681 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:26:58.087681 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:26:58.087681 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:26:58.087681 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:26:58.087681 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:26:58.087681 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:26:58.088568 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:26:58.088568 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:26:58.088568 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:26:58.088568 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:26:58.088568 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 08:26:58.088568 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 08:26:58.088568 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 08:26:58.088568 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 08:26:58.088568 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 08:26:58.088568 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 08:26:58.088568 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 08:26:58.088568 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:26:58.088568 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 629/1249] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Aug 10 08:26:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:26:58.094805 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a171e7ea-53aa-4f8a-9c24-8e2a04361593 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:26:58.193769 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a171e7ea-53aa-4f8a-9c24-8e2a04361593 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:26:58.223813 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:26:58.228501 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7bfccf-86ab-45e9-83e6-439bbefdfc9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:26:58.229127 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7bfccf-86ab-45e9-83e6-439bbefdfc9f None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:26:58.229404 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7bfccf-86ab-45e9-83e6-439bbefdfc9f None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:26:58.229667 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7bfccf-86ab-45e9-83e6-439bbefdfc9f None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:26:58.230312 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7bfccf-86ab-45e9-83e6-439bbefdfc9f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:26:58.230608 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7bfccf-86ab-45e9-83e6-439bbefdfc9f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:26:58.231949 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7bfccf-86ab-45e9-83e6-439bbefdfc9f None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:26:58.240170 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 621/1250] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Aug 10 08:26:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:26:58.246182 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf69cc0-6766-4237-aee6-f37b5fac6997 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:26:58.323571 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:26:58.346429 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf69cc0-6766-4237-aee6-f37b5fac6997 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:26:58.378899 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed9203b-17a5-4ef3-9578-102c70376d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:26:58.379510 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed9203b-17a5-4ef3-9578-102c70376d50 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:26:58.379785 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed9203b-17a5-4ef3-9578-102c70376d50 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:26:58.380052 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed9203b-17a5-4ef3-9578-102c70376d50 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:26:58.390333 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed9203b-17a5-4ef3-9578-102c70376d50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:26:58.392259 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed9203b-17a5-4ef3-9578-102c70376d50 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:26:58.407485 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-1ed9203b-17a5-4ef3-9578-102c70376d50 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:26:58.407485 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:26:58.407485 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:26:58.407485 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:26:58.407485 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:26:58.407485 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:26:58.407485 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:26:58.407485 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:26:58.407485 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:26:58.407485 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:26:58.407485 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:26:58.407485 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:26:58.407485 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 08:26:58.407485 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 08:26:58.407485 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 08:26:58.407485 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 08:26:58.408635 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:26:58.408635 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:26:58.408635 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:26:58.408635 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:26:58.408635 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:26:58.408635 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:26:58.408635 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:26:58.408635 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:26:58.408635 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:26:58.408635 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:26:58.408635 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:26:58.408635 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:26:58.408635 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:26:58.408635 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:26:58.408635 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:26:58.408635 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:26:58.408635 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 08:26:58.409959 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 08:26:58.409959 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:26:58.409959 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:26:58.409959 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 630/1251] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:26:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:26:58.414642 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7bfccf-86ab-45e9-83e6-439bbefdfc9f None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:26:58.512697 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7bfccf-86ab-45e9-83e6-439bbefdfc9f None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:26:58.546724 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b945f9a1-281f-4bd5-961f-f29057ba48cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:26:58.547364 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b945f9a1-281f-4bd5-961f-f29057ba48cf None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:26:58.547705 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b945f9a1-281f-4bd5-961f-f29057ba48cf None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:26:58.548006 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b945f9a1-281f-4bd5-961f-f29057ba48cf None admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:26:58.548638 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b945f9a1-281f-4bd5-961f-f29057ba48cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:26:58.548961 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b945f9a1-281f-4bd5-961f-f29057ba48cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:26:58.550318 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b945f9a1-281f-4bd5-961f-f29057ba48cf None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:26:58.561750 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 622/1252] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Aug 10 08:26:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:26:58.568221 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed9203b-17a5-4ef3-9578-102c70376d50 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:26:58.665861 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed9203b-17a5-4ef3-9578-102c70376d50 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:26:58.691807 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:26:58.699499 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f1ebfa-07e0-4b59-af3d-0d80a0f3bb29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:26:58.700120 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f1ebfa-07e0-4b59-af3d-0d80a0f3bb29 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:26:58.700397 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f1ebfa-07e0-4b59-af3d-0d80a0f3bb29 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:26:58.700667 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f1ebfa-07e0-4b59-af3d-0d80a0f3bb29 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:26:58.708595 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f1ebfa-07e0-4b59-af3d-0d80a0f3bb29 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:26:58.712791 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f1ebfa-07e0-4b59-af3d-0d80a0f3bb29 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:26:58.720725 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-a5f1ebfa-07e0-4b59-af3d-0d80a0f3bb29 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 10 08:26:58.720725 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:26:58.720725 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:26:58.720725 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:26:58.720725 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:26:58.720725 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:26:58.720725 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:26:58.720725 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:26:58.720725 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:26:58.720725 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:26:58.720725 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:26:58.720725 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:26:58.720725 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 08:26:58.720725 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 08:26:58.720725 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 08:26:58.720725 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 08:26:58.721903 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:26:58.721903 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:26:58.721903 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 08:26:58.721903 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 08:26:58.721903 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 08:26:58.721903 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 08:26:58.721903 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:26:58.721903 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:26:58.721903 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:26:58.721903 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:26:58.721903 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:26:58.721903 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:26:58.721903 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:26:58.721903 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:26:58.721903 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:26:58.721903 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:26:58.721903 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:26:58.721903 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:26:58.723106 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:26:58.723106 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:26:58.723106 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 08:26:58.723106 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 08:26:58.723106 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 08:26:58.723106 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 08:26:58.723106 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 08:26:58.723106 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 08:26:58.723106 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 10 08:26:58.723106 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:26:58.723106 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 631/1253] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:26:58 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:26:58.727670 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b945f9a1-281f-4bd5-961f-f29057ba48cf None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:26:58.785803 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:26:58.829233 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b945f9a1-281f-4bd5-961f-f29057ba48cf None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:26:58.862595 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570ea571-dc23-4497-95d0-ceed189a1a8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:26:58.863186 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570ea571-dc23-4497-95d0-ceed189a1a8c None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:26:58.863464 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570ea571-dc23-4497-95d0-ceed189a1a8c None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:26:58.863731 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570ea571-dc23-4497-95d0-ceed189a1a8c None admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:26:58.864456 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570ea571-dc23-4497-95d0-ceed189a1a8c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:26:58.864756 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570ea571-dc23-4497-95d0-ceed189a1a8c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:26:58.866215 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570ea571-dc23-4497-95d0-ceed189a1a8c None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:26:58.884187 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 623/1254] 10.222.0.57 () {56 vars in 1143 bytes} [Wed Aug 10 08:26:58 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:26:58.890068 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f1ebfa-07e0-4b59-af3d-0d80a0f3bb29 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:26:58.988963 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f1ebfa-07e0-4b59-af3d-0d80a0f3bb29 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:26:59.023301 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d4cbfd-5086-4ae5-8e2d-f2d9d8df8585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:26:59.023901 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d4cbfd-5086-4ae5-8e2d-f2d9d8df8585 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:26:59.024181 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d4cbfd-5086-4ae5-8e2d-f2d9d8df8585 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:26:59.024453 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d4cbfd-5086-4ae5-8e2d-f2d9d8df8585 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:26:59.078153 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:26:59.089869 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d4cbfd-5086-4ae5-8e2d-f2d9d8df8585 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:26:59.091324 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d4cbfd-5086-4ae5-8e2d-f2d9d8df8585 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:26:59.189421 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 632/1255] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Aug 10 08:26:58 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 08:26:59.195463 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570ea571-dc23-4497-95d0-ceed189a1a8c None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:26:59.293511 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570ea571-dc23-4497-95d0-ceed189a1a8c None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:26:59.306440 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:26:59.327305 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87b0f65-e264-4b4f-bb68-ce05d41b3399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:26:59.328529 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87b0f65-e264-4b4f-bb68-ce05d41b3399 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:26:59.328813 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87b0f65-e264-4b4f-bb68-ce05d41b3399 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:26:59.329116 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87b0f65-e264-4b4f-bb68-ce05d41b3399 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:26:59.338714 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87b0f65-e264-4b4f-bb68-ce05d41b3399 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:26:59.340127 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87b0f65-e264-4b4f-bb68-ce05d41b3399 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:26:59.354164 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-a87b0f65-e264-4b4f-bb68-ce05d41b3399 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:26:59.354164 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:26:59.354164 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:26:59.354164 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:26:59.354164 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:26:59.354164 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:26:59.354164 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:26:59.354164 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:26:59.354164 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:26:59.354164 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:26:59.354164 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:26:59.354164 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:26:59.354164 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 08:26:59.354164 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 08:26:59.354164 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 08:26:59.354164 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 08:26:59.355289 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:26:59.355289 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:26:59.355289 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:26:59.355289 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:26:59.355289 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:26:59.355289 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:26:59.355289 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:26:59.355289 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:26:59.355289 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:26:59.355289 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:26:59.355289 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:26:59.355289 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:26:59.355289 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:26:59.355289 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:26:59.355289 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:26:59.355289 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:26:59.355289 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 08:26:59.356427 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 08:26:59.356427 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:26:59.356427 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:26:59.356427 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 624/1256] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:26:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:26:59.361066 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d4cbfd-5086-4ae5-8e2d-f2d9d8df8585 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:26:59.451691 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:26:59.465511 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d4cbfd-5086-4ae5-8e2d-f2d9d8df8585 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:26:59.498701 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d035ee-4209-456e-927f-b10fa39ee370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:26:59.499669 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d035ee-4209-456e-927f-b10fa39ee370 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:26:59.500513 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d035ee-4209-456e-927f-b10fa39ee370 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:26:59.500787 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d035ee-4209-456e-927f-b10fa39ee370 None admin] PATH_INFO: `/v3/domains` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:26:59.501571 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d035ee-4209-456e-927f-b10fa39ee370 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:26:59.501877 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d035ee-4209-456e-927f-b10fa39ee370 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:26:59.503275 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d035ee-4209-456e-927f-b10fa39ee370 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:26:59.514790 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 633/1257] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Aug 10 08:26:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:26:59.521240 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87b0f65-e264-4b4f-bb68-ce05d41b3399 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:26:59.622158 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87b0f65-e264-4b4f-bb68-ce05d41b3399 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:26:59.655739 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73dba28-0db1-44aa-be96-070cc8a47140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:26:59.656548 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73dba28-0db1-44aa-be96-070cc8a47140 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:26:59.656837 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73dba28-0db1-44aa-be96-070cc8a47140 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:26:59.657135 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73dba28-0db1-44aa-be96-070cc8a47140 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:26:59.663598 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73dba28-0db1-44aa-be96-070cc8a47140 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:26:59.665034 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73dba28-0db1-44aa-be96-070cc8a47140 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:26:59.676448 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-e73dba28-0db1-44aa-be96-070cc8a47140 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 08:26:59.676448 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:26:59.676448 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:26:59.676448 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:26:59.676448 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:26:59.676448 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:26:59.676448 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:26:59.676448 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:26:59.676448 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:26:59.676448 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:26:59.676448 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:26:59.676448 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:26:59.676448 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 08:26:59.676448 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 08:26:59.676448 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 08:26:59.677540 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 08:26:59.677540 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:26:59.677540 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:26:59.677540 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:26:59.677540 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:26:59.677540 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:26:59.677540 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:26:59.677540 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:26:59.677540 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:26:59.677540 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:26:59.677540 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:26:59.677540 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:26:59.677540 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:26:59.677540 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:26:59.677540 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:26:59.677540 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:26:59.677540 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:26:59.678621 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 08:26:59.678621 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 08:26:59.678621 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 08:26:59.678621 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 08:26:59.678621 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 08:26:59.678621 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 08:26:59.678621 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 08:26:59.678621 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:26:59.678621 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 625/1258] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Aug 10 08:26:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:26:59.683565 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d035ee-4209-456e-927f-b10fa39ee370 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:26:59.784891 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d035ee-4209-456e-927f-b10fa39ee370 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:26:59.819075 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db041421-0f34-4e9f-b6f4-8b45576eb819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:26:59.819679 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db041421-0f34-4e9f-b6f4-8b45576eb819 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:26:59.819958 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db041421-0f34-4e9f-b6f4-8b45576eb819 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:26:59.820228 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db041421-0f34-4e9f-b6f4-8b45576eb819 None admin] PATH_INFO: `/v3/projects` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:26:59.820955 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db041421-0f34-4e9f-b6f4-8b45576eb819 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:26:59.821271 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db041421-0f34-4e9f-b6f4-8b45576eb819 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:26:59.822693 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db041421-0f34-4e9f-b6f4-8b45576eb819 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:26:59.834789 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 634/1259] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Aug 10 08:26:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:26:59.841008 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73dba28-0db1-44aa-be96-070cc8a47140 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:26:59.867481 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:26:59.941289 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73dba28-0db1-44aa-be96-070cc8a47140 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:26:59.975497 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec5ebf0-8455-4a45-8caa-ac31afe46db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:26:59.976093 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec5ebf0-8455-4a45-8caa-ac31afe46db3 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:26:59.976373 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec5ebf0-8455-4a45-8caa-ac31afe46db3 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:26:59.976644 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec5ebf0-8455-4a45-8caa-ac31afe46db3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:26:59.977402 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec5ebf0-8455-4a45-8caa-ac31afe46db3 None admin] RBAC: Adding query filter params (role.id=ec3751692d9c4ecf81b4a71c936c46be, scope.project.id=b49e381419e940278bcb6111899589b5, user.id=2912307c56594125b157f1cfdf7c96a9) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:26:59.977710 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec5ebf0-8455-4a45-8caa-ac31afe46db3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:26:59.979127 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec5ebf0-8455-4a45-8caa-ac31afe46db3 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:26:59.985322 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 626/1260] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Aug 10 08:26:59 2022] GET /identity/v3/role_assignments?user.id=2912307c56594125b157f1cfdf7c96a9&scope.project.id=b49e381419e940278bcb6111899589b5&role.id=ec3751692d9c4ecf81b4a71c936c46be => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:27:01.951769 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783bab17-2db9-4c23-b39c-19a5995d6949 None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:27:01.952326 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783bab17-2db9-4c23-b39c-19a5995d6949 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:27:01.952326 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783bab17-2db9-4c23-b39c-19a5995d6949 None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:27:01.953787 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 635/1261] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:27:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:27:01.959045 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccdd148-abc4-4b04-8545-9285238399cd None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:27:01.959437 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccdd148-abc4-4b04-8545-9285238399cd None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:27:01.959581 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccdd148-abc4-4b04-8545-9285238399cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:27:01.977976 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-3ccdd148-abc4-4b04-8545-9285238399cd None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:27:01.979196 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-3ccdd148-abc4-4b04-8545-9285238399cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:27:02.044220 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 627/1262] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:27:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 08:27:02.067880 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783bab17-2db9-4c23-b39c-19a5995d6949 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:27:02.078700 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:27:02.168647 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783bab17-2db9-4c23-b39c-19a5995d6949 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:27:02.203488 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61efcc31-10da-4417-80b8-bd3047e0f546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:27:02.203683 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61efcc31-10da-4417-80b8-bd3047e0f546 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:27:02.203944 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61efcc31-10da-4417-80b8-bd3047e0f546 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:27:02.204213 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61efcc31-10da-4417-80b8-bd3047e0f546 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:27:02.214075 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61efcc31-10da-4417-80b8-bd3047e0f546 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 08:27:02.219143 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-61efcc31-10da-4417-80b8-bd3047e0f546 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 08:27:02.219143 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:27:02.219143 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:27:02.219143 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:27:02.219143 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:27:02.219143 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:27:02.219143 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:27:02.219143 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:27:02.219143 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:27:02.219143 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:27:02.219143 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:27:02.219143 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:27:02.219143 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 08:27:02.219143 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 08:27:02.219143 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 08:27:02.219143 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 08:27:02.220462 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 08:27:02.220462 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 08:27:02.220462 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 08:27:02.220462 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 08:27:02.220462 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:27:02.220462 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:27:02.220462 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:27:02.220462 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:27:02.220462 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:27:02.220462 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:27:02.220462 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:27:02.220462 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:27:02.220462 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:27:02.220462 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:27:02.220462 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:27:02.220462 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:27:02.221660 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:27:02.221660 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:27:02.221660 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:27:02.221660 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:27:02.221660 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 08:27:02.221660 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 08:27:02.221660 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 08:27:02.221660 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 08:27:02.221660 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 08:27:02.221660 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 08:27:02.221660 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 08:27:02.221660 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:27:02.221660 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 636/1263] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Aug 10 08:27:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:27:02.226196 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccdd148-abc4-4b04-8545-9285238399cd None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:27:02.245695 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:27:02.326723 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccdd148-abc4-4b04-8545-9285238399cd None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:27:02.361284 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d91bc8-c888-4833-99fb-2a5066385fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:27:02.361873 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d91bc8-c888-4833-99fb-2a5066385fac None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:27:02.362149 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d91bc8-c888-4833-99fb-2a5066385fac None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:27:02.362412 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d91bc8-c888-4833-99fb-2a5066385fac None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:27:02.363060 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d91bc8-c888-4833-99fb-2a5066385fac None admin] RBAC: Adding query filter params (name=admin) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:27:02.363356 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d91bc8-c888-4833-99fb-2a5066385fac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:27:02.364734 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d91bc8-c888-4833-99fb-2a5066385fac None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:27:02.372709 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 628/1264] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Aug 10 08:27:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:27:02.379144 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61efcc31-10da-4417-80b8-bd3047e0f546 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:27:02.446938 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:27:02.478437 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61efcc31-10da-4417-80b8-bd3047e0f546 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:27:02.511608 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc42ebdc-a074-42fa-bf23-ee1423d5cca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:27:02.512229 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc42ebdc-a074-42fa-bf23-ee1423d5cca8 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:27:02.512500 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc42ebdc-a074-42fa-bf23-ee1423d5cca8 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:27:02.512766 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc42ebdc-a074-42fa-bf23-ee1423d5cca8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:27:02.522643 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc42ebdc-a074-42fa-bf23-ee1423d5cca8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:27:02.524070 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc42ebdc-a074-42fa-bf23-ee1423d5cca8 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:27:02.539560 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-dc42ebdc-a074-42fa-bf23-ee1423d5cca8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:27:02.539560 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:27:02.539560 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:27:02.539560 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:27:02.539560 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:27:02.539560 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:27:02.539560 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:27:02.539560 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:27:02.539560 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:27:02.539560 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:27:02.539560 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:27:02.539560 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:27:02.539560 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 08:27:02.539560 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 08:27:02.539560 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 08:27:02.539560 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 08:27:02.540830 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:27:02.540830 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:27:02.540830 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:27:02.540830 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:27:02.540830 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:27:02.540830 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:27:02.540830 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:27:02.540830 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:27:02.540830 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:27:02.540830 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:27:02.540830 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:27:02.540830 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:27:02.540830 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:27:02.540830 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:27:02.540830 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:27:02.540830 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:27:02.540830 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 08:27:02.542032 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 08:27:02.542032 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:27:02.542032 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:27:02.542032 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 637/1265] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:27:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:27:02.546703 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d91bc8-c888-4833-99fb-2a5066385fac None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:27:02.626502 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:27:02.645582 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d91bc8-c888-4833-99fb-2a5066385fac None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:27:02.678530 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5677362d-e007-4368-b166-0a5dd912dc16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:27:02.679130 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5677362d-e007-4368-b166-0a5dd912dc16 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:27:02.679410 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5677362d-e007-4368-b166-0a5dd912dc16 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:27:02.679681 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5677362d-e007-4368-b166-0a5dd912dc16 None admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:27:02.680318 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5677362d-e007-4368-b166-0a5dd912dc16 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:27:02.680621 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5677362d-e007-4368-b166-0a5dd912dc16 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:27:02.682013 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5677362d-e007-4368-b166-0a5dd912dc16 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:27:02.694190 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 629/1266] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Aug 10 08:27:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:27:02.700006 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc42ebdc-a074-42fa-bf23-ee1423d5cca8 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:27:02.800834 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc42ebdc-a074-42fa-bf23-ee1423d5cca8 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:27:02.820285 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:27:02.834368 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a2fcd4-7761-472b-b421-7024e153fe22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:27:02.834994 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a2fcd4-7761-472b-b421-7024e153fe22 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:27:02.835274 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a2fcd4-7761-472b-b421-7024e153fe22 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:27:02.835549 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a2fcd4-7761-472b-b421-7024e153fe22 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:27:02.842902 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a2fcd4-7761-472b-b421-7024e153fe22 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:27:02.845078 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a2fcd4-7761-472b-b421-7024e153fe22 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:27:02.855885 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-23a2fcd4-7761-472b-b421-7024e153fe22 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 10 08:27:02.855885 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:27:02.855885 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:27:02.855885 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:27:02.855885 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:27:02.855885 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:27:02.855885 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:27:02.855885 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:27:02.855885 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:27:02.855885 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:27:02.855885 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:27:02.855885 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:27:02.855885 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 08:27:02.855885 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 08:27:02.855885 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 08:27:02.855885 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 08:27:02.861796 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:27:02.861796 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:27:02.861796 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 08:27:02.861796 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 08:27:02.861796 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 08:27:02.861796 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 08:27:02.861796 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:27:02.861796 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:27:02.861796 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:27:02.861796 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:27:02.861796 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:27:02.861796 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:27:02.861796 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:27:02.861796 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:27:02.861796 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:27:02.861796 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:27:02.861796 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:27:02.861796 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:27:02.862968 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:27:02.862968 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:27:02.862968 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 08:27:02.862968 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 08:27:02.862968 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 08:27:02.862968 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 08:27:02.862968 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 08:27:02.862968 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 08:27:02.862968 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 10 08:27:02.862968 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:27:02.862968 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 638/1267] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:27:02 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:27:02.863727 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5677362d-e007-4368-b166-0a5dd912dc16 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:27:02.962386 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5677362d-e007-4368-b166-0a5dd912dc16 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:27:02.997154 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b0a552-b027-4eb7-b515-786d647b94bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:27:02.997692 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b0a552-b027-4eb7-b515-786d647b94bd None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:27:02.997968 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b0a552-b027-4eb7-b515-786d647b94bd None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:27:02.998234 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b0a552-b027-4eb7-b515-786d647b94bd None admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:27:02.998953 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b0a552-b027-4eb7-b515-786d647b94bd None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:27:02.999255 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b0a552-b027-4eb7-b515-786d647b94bd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:27:03.000799 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b0a552-b027-4eb7-b515-786d647b94bd None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:27:03.009204 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:27:03.018102 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 630/1268] 10.222.0.57 () {56 vars in 1143 bytes} [Wed Aug 10 08:27:02 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:27:03.023791 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a2fcd4-7761-472b-b421-7024e153fe22 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:27:03.122321 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a2fcd4-7761-472b-b421-7024e153fe22 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:27:03.155808 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538a709b-4e15-45ce-bbcb-95d7ffb4edb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:27:03.156446 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538a709b-4e15-45ce-bbcb-95d7ffb4edb0 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:27:03.156791 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538a709b-4e15-45ce-bbcb-95d7ffb4edb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:27:03.157109 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538a709b-4e15-45ce-bbcb-95d7ffb4edb0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:27:03.167506 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538a709b-4e15-45ce-bbcb-95d7ffb4edb0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:27:03.168967 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538a709b-4e15-45ce-bbcb-95d7ffb4edb0 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:27:03.182957 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-538a709b-4e15-45ce-bbcb-95d7ffb4edb0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:27:03.182957 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:27:03.182957 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:27:03.182957 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:27:03.182957 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:27:03.182957 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:27:03.182957 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:27:03.182957 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:27:03.182957 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:27:03.182957 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:27:03.182957 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:27:03.182957 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:27:03.182957 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 08:27:03.182957 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 08:27:03.182957 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 08:27:03.182957 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 08:27:03.184101 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:27:03.184101 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:27:03.184101 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:27:03.184101 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:27:03.184101 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:27:03.184101 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:27:03.184101 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:27:03.184101 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:27:03.184101 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:27:03.184101 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:27:03.184101 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:27:03.184101 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:27:03.184101 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:27:03.184101 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:27:03.184101 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:27:03.184101 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:27:03.184101 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 08:27:03.185227 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 08:27:03.185227 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:27:03.185227 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:27:03.185227 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 639/1269] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:27:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:27:03.190656 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b0a552-b027-4eb7-b515-786d647b94bd None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:27:03.292421 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b0a552-b027-4eb7-b515-786d647b94bd None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:27:03.326348 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76895e3-ef3f-48f3-8dfc-349b0ad63180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:27:03.326965 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76895e3-ef3f-48f3-8dfc-349b0ad63180 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:27:03.327240 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76895e3-ef3f-48f3-8dfc-349b0ad63180 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:27:03.327505 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76895e3-ef3f-48f3-8dfc-349b0ad63180 None admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:27:03.328825 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76895e3-ef3f-48f3-8dfc-349b0ad63180 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:27:03.329559 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76895e3-ef3f-48f3-8dfc-349b0ad63180 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:27:03.330929 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76895e3-ef3f-48f3-8dfc-349b0ad63180 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:27:03.341662 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 631/1270] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Aug 10 08:27:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:27:03.347626 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538a709b-4e15-45ce-bbcb-95d7ffb4edb0 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:27:03.379914 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:27:03.448435 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538a709b-4e15-45ce-bbcb-95d7ffb4edb0 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:27:03.482767 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6479363-94b4-49ee-ad87-5569c566113c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:27:03.483395 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6479363-94b4-49ee-ad87-5569c566113c None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:27:03.483673 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6479363-94b4-49ee-ad87-5569c566113c None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:27:03.483939 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6479363-94b4-49ee-ad87-5569c566113c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:27:03.490867 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6479363-94b4-49ee-ad87-5569c566113c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:27:03.492300 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6479363-94b4-49ee-ad87-5569c566113c None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:27:03.504023 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-a6479363-94b4-49ee-ad87-5569c566113c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 08:27:03.504023 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:27:03.504023 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:27:03.504023 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:27:03.504023 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:27:03.504023 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:27:03.504023 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:27:03.504023 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:27:03.504023 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:27:03.504023 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:27:03.504023 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:27:03.504023 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:27:03.504023 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 08:27:03.504023 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 08:27:03.504023 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 08:27:03.505038 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 08:27:03.505038 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:27:03.505038 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:27:03.505038 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:27:03.505038 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:27:03.505038 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:27:03.505038 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:27:03.505038 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:27:03.505038 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:27:03.505038 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:27:03.505038 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:27:03.505038 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:27:03.505038 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:27:03.505038 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:27:03.505038 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:27:03.505038 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:27:03.505038 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:27:03.506007 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 08:27:03.506007 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 08:27:03.506007 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 08:27:03.506007 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 08:27:03.506007 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 08:27:03.506007 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 08:27:03.506007 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 08:27:03.506007 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:27:03.506007 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 640/1271] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Aug 10 08:27:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:27:03.511135 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76895e3-ef3f-48f3-8dfc-349b0ad63180 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:27:03.569040 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:27:03.610028 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76895e3-ef3f-48f3-8dfc-349b0ad63180 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:27:03.643985 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99b509a-ad12-441b-a5fa-8d491a1ae9b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:27:03.644540 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99b509a-ad12-441b-a5fa-8d491a1ae9b9 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:27:03.644820 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99b509a-ad12-441b-a5fa-8d491a1ae9b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:27:03.645124 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99b509a-ad12-441b-a5fa-8d491a1ae9b9 None admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:27:03.645814 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99b509a-ad12-441b-a5fa-8d491a1ae9b9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:27:03.646120 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99b509a-ad12-441b-a5fa-8d491a1ae9b9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:27:03.647541 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99b509a-ad12-441b-a5fa-8d491a1ae9b9 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:27:03.659997 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 632/1272] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Aug 10 08:27:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:27:03.667525 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6479363-94b4-49ee-ad87-5569c566113c None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:27:03.760449 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:27:03.769496 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6479363-94b4-49ee-ad87-5569c566113c None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:27:03.802460 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb94843-0405-4f4d-9ce7-b0a0b7d65cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:27:03.803116 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb94843-0405-4f4d-9ce7-b0a0b7d65cd0 None admin] REQUEST_METHOD: `PUT` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:27:03.803396 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb94843-0405-4f4d-9ce7-b0a0b7d65cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:27:03.803669 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb94843-0405-4f4d-9ce7-b0a0b7d65cd0 None admin] PATH_INFO: `/v3/projects/b49e381419e940278bcb6111899589b5/users/2912307c56594125b157f1cfdf7c96a9/roles/ec3751692d9c4ecf81b4a71c936c46be` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:27:03.805670 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb94843-0405-4f4d-9ce7-b0a0b7d65cd0 None admin] RBAC: Authorizing `identity:create_grant(project_id=b49e381419e940278bcb6111899589b5, user_id=2912307c56594125b157f1cfdf7c96a9, role_id=ec3751692d9c4ecf81b4a71c936c46be)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:27:03.807108 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb94843-0405-4f4d-9ce7-b0a0b7d65cd0 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:27:03.868823 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 641/1273] 10.222.0.57 () {58 vars in 1439 bytes} [Wed Aug 10 08:27:03 2022] PUT /identity/v3/projects/b49e381419e940278bcb6111899589b5/users/2912307c56594125b157f1cfdf7c96a9/roles/ec3751692d9c4ecf81b4a71c936c46be => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 08:27:05.857247 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2aa9db-f3b2-450a-bf76-a7939a845fb2 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:27:05.857827 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2aa9db-f3b2-450a-bf76-a7939a845fb2 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:27:05.857827 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2aa9db-f3b2-450a-bf76-a7939a845fb2 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:27:05.859154 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 633/1274] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:27:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:27:05.864660 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca169e3-0c47-4602-8a9e-7af9b0732b11 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:27:05.865154 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca169e3-0c47-4602-8a9e-7af9b0732b11 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:27:05.865251 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca169e3-0c47-4602-8a9e-7af9b0732b11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:27:05.883273 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-fca169e3-0c47-4602-8a9e-7af9b0732b11 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:27:05.884457 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-fca169e3-0c47-4602-8a9e-7af9b0732b11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:27:05.948455 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 642/1275] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:27:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 08:27:05.972308 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2aa9db-f3b2-450a-bf76-a7939a845fb2 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:27:06.083700 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2aa9db-f3b2-450a-bf76-a7939a845fb2 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:27:06.101739 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:27:06.119208 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20608f7-ffcb-490a-bea7-369a82247523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:27:06.119789 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20608f7-ffcb-490a-bea7-369a82247523 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:27:06.120065 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20608f7-ffcb-490a-bea7-369a82247523 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:27:06.120332 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20608f7-ffcb-490a-bea7-369a82247523 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:27:06.131685 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20608f7-ffcb-490a-bea7-369a82247523 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 08:27:06.136419 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-e20608f7-ffcb-490a-bea7-369a82247523 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 08:27:06.136419 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:27:06.136419 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:27:06.136419 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:27:06.136419 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:27:06.136419 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:27:06.136419 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:27:06.136419 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:27:06.136419 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:27:06.136419 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:27:06.136419 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:27:06.136419 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:27:06.136419 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 08:27:06.136419 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 08:27:06.136419 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 08:27:06.136419 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 08:27:06.142271 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 08:27:06.142271 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 08:27:06.142271 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 08:27:06.142271 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 08:27:06.142271 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:27:06.142271 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:27:06.142271 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:27:06.142271 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:27:06.142271 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:27:06.142271 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:27:06.142271 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:27:06.142271 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:27:06.142271 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:27:06.142271 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:27:06.142271 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:27:06.142271 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:27:06.143277 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:27:06.143277 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:27:06.143277 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:27:06.143277 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:27:06.143277 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 08:27:06.143277 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 08:27:06.143277 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 08:27:06.143277 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 08:27:06.143277 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 08:27:06.143277 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 08:27:06.143277 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 08:27:06.143277 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:27:06.143277 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 634/1276] 10.222.0.57 () {56 vars in 1096 bytes} [Wed Aug 10 08:27:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 08:27:06.144018 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca169e3-0c47-4602-8a9e-7af9b0732b11 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:27:06.225178 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:27:06.243348 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca169e3-0c47-4602-8a9e-7af9b0732b11 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:27:06.276893 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7443bca-84db-4730-b0c7-37e6c25076aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:27:06.277522 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7443bca-84db-4730-b0c7-37e6c25076aa None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:27:06.277802 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7443bca-84db-4730-b0c7-37e6c25076aa None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:27:06.278074 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7443bca-84db-4730-b0c7-37e6c25076aa None admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:27:06.278761 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7443bca-84db-4730-b0c7-37e6c25076aa None admin] RBAC: Adding query filter params (name=admin) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:27:06.279113 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7443bca-84db-4730-b0c7-37e6c25076aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:27:06.281573 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7443bca-84db-4730-b0c7-37e6c25076aa None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:27:06.289425 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 643/1277] 10.222.0.57 () {56 vars in 1099 bytes} [Wed Aug 10 08:27:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:27:06.295263 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20608f7-ffcb-490a-bea7-369a82247523 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:27:06.394926 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20608f7-ffcb-490a-bea7-369a82247523 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:27:06.430231 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e61207d-e031-4671-b096-8773c47bbf1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:27:06.430854 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e61207d-e031-4671-b096-8773c47bbf1e None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:27:06.431136 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e61207d-e031-4671-b096-8773c47bbf1e None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:27:06.431407 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e61207d-e031-4671-b096-8773c47bbf1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:27:06.441899 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e61207d-e031-4671-b096-8773c47bbf1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:27:06.443332 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e61207d-e031-4671-b096-8773c47bbf1e None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:27:06.458447 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-0e61207d-e031-4671-b096-8773c47bbf1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:27:06.458447 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:27:06.458447 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:27:06.458447 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:27:06.458447 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:27:06.458447 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:27:06.458447 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:27:06.458447 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:27:06.458447 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:27:06.458447 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:27:06.458447 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:27:06.458447 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:27:06.458447 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 08:27:06.458447 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 08:27:06.458447 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 08:27:06.458447 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 08:27:06.459445 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:27:06.459445 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:27:06.459445 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:27:06.459445 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:27:06.459445 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:27:06.459445 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:27:06.459445 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:27:06.459445 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:27:06.459445 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:27:06.459445 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:27:06.459445 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:27:06.459445 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:27:06.459445 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:27:06.459445 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:27:06.459445 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:27:06.459445 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:27:06.459445 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 08:27:06.460465 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 08:27:06.460465 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:27:06.460465 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:27:06.460465 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 635/1278] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:27:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:27:06.465441 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7443bca-84db-4730-b0c7-37e6c25076aa None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:27:06.571858 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7443bca-84db-4730-b0c7-37e6c25076aa None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:27:06.603811 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7a869e-5ee6-496f-b7e7-00ae8d60f7b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:27:06.604401 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7a869e-5ee6-496f-b7e7-00ae8d60f7b5 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:27:06.604706 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7a869e-5ee6-496f-b7e7-00ae8d60f7b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:27:06.605028 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7a869e-5ee6-496f-b7e7-00ae8d60f7b5 None admin] PATH_INFO: `/v3/domains` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:27:06.605659 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7a869e-5ee6-496f-b7e7-00ae8d60f7b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:27:06.605957 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7a869e-5ee6-496f-b7e7-00ae8d60f7b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:27:06.607306 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7a869e-5ee6-496f-b7e7-00ae8d60f7b5 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:27:06.619925 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 644/1279] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Aug 10 08:27:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:27:06.626344 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e61207d-e031-4671-b096-8773c47bbf1e None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:27:06.656950 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:27:06.724774 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e61207d-e031-4671-b096-8773c47bbf1e None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:27:06.758188 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8327719-2696-44e8-8678-a196e310f144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:27:06.758804 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8327719-2696-44e8-8678-a196e310f144 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:27:06.759082 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8327719-2696-44e8-8678-a196e310f144 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:27:06.759354 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8327719-2696-44e8-8678-a196e310f144 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:27:06.766899 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8327719-2696-44e8-8678-a196e310f144 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:27:06.768322 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8327719-2696-44e8-8678-a196e310f144 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:27:06.779067 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-b8327719-2696-44e8-8678-a196e310f144 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 10 08:27:06.779067 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:27:06.779067 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:27:06.779067 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:27:06.779067 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:27:06.779067 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:27:06.779067 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:27:06.779067 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:27:06.779067 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:27:06.779067 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:27:06.779067 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:27:06.779067 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:27:06.779067 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 08:27:06.779067 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 08:27:06.779067 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 08:27:06.779067 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 08:27:06.780255 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:27:06.780255 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:27:06.780255 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 08:27:06.780255 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 08:27:06.780255 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 08:27:06.780255 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 08:27:06.780255 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:27:06.780255 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:27:06.780255 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:27:06.780255 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:27:06.780255 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:27:06.780255 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:27:06.780255 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:27:06.780255 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:27:06.780255 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:27:06.780255 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:27:06.780255 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:27:06.780255 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:27:06.781826 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:27:06.781826 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:27:06.781826 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 08:27:06.781826 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 08:27:06.781826 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 08:27:06.781826 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 08:27:06.781826 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 08:27:06.781826 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 08:27:06.781826 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 10 08:27:06.781826 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:27:06.781826 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 636/1280] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:27:06 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:27:06.786841 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7a869e-5ee6-496f-b7e7-00ae8d60f7b5 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:27:06.790523 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:27:06.884825 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7a869e-5ee6-496f-b7e7-00ae8d60f7b5 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:27:06.917626 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aff7783-a3cf-4f32-9f18-16ee61336dac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:27:06.918219 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aff7783-a3cf-4f32-9f18-16ee61336dac None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:27:06.918496 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aff7783-a3cf-4f32-9f18-16ee61336dac None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:27:06.918761 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aff7783-a3cf-4f32-9f18-16ee61336dac None admin] PATH_INFO: `/v3/users` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:27:06.919478 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aff7783-a3cf-4f32-9f18-16ee61336dac None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:27:06.919778 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aff7783-a3cf-4f32-9f18-16ee61336dac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:27:06.921220 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aff7783-a3cf-4f32-9f18-16ee61336dac None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:27:06.938152 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 645/1281] 10.222.0.57 () {56 vars in 1143 bytes} [Wed Aug 10 08:27:06 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:27:06.944099 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8327719-2696-44e8-8678-a196e310f144 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:27:07.034423 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:27:07.043571 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8327719-2696-44e8-8678-a196e310f144 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:27:07.076107 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daa850b-a90e-491e-8d6f-e4e6a5d34649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:27:07.076695 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daa850b-a90e-491e-8d6f-e4e6a5d34649 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:27:07.076995 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daa850b-a90e-491e-8d6f-e4e6a5d34649 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:27:07.077264 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daa850b-a90e-491e-8d6f-e4e6a5d34649 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:27:07.145637 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daa850b-a90e-491e-8d6f-e4e6a5d34649 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:27:07.147072 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daa850b-a90e-491e-8d6f-e4e6a5d34649 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:27:07.234538 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:27:07.244735 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 637/1282] 10.222.0.57 () {58 vars in 1282 bytes} [Wed Aug 10 08:27:06 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 08:27:07.250977 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aff7783-a3cf-4f32-9f18-16ee61336dac None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:27:07.313991 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:27:07.351731 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aff7783-a3cf-4f32-9f18-16ee61336dac None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:27:07.384410 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6d86ae-0d1e-41ff-ba5c-d4581cf1df61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:27:07.385076 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6d86ae-0d1e-41ff-ba5c-d4581cf1df61 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:27:07.385351 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6d86ae-0d1e-41ff-ba5c-d4581cf1df61 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:27:07.385617 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6d86ae-0d1e-41ff-ba5c-d4581cf1df61 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:27:07.396666 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6d86ae-0d1e-41ff-ba5c-d4581cf1df61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:27:07.399219 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6d86ae-0d1e-41ff-ba5c-d4581cf1df61 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:27:07.413076 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-7c6d86ae-0d1e-41ff-ba5c-d4581cf1df61 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:27:07.413076 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:27:07.413076 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:27:07.413076 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:27:07.413076 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:27:07.413076 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:27:07.413076 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:27:07.413076 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:27:07.413076 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:27:07.413076 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:27:07.413076 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:27:07.413076 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:27:07.413076 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 08:27:07.413076 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 08:27:07.413076 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 08:27:07.413076 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 08:27:07.414274 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:27:07.414274 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:27:07.414274 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:27:07.414274 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:27:07.414274 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:27:07.414274 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:27:07.414274 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:27:07.414274 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:27:07.414274 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:27:07.414274 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:27:07.414274 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:27:07.414274 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:27:07.414274 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:27:07.414274 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:27:07.414274 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:27:07.414274 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:27:07.414274 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 08:27:07.415226 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 08:27:07.415226 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 08:27:07.415226 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:27:07.415226 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 646/1283] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:27:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:27:07.419952 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daa850b-a90e-491e-8d6f-e4e6a5d34649 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:27:07.516628 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daa850b-a90e-491e-8d6f-e4e6a5d34649 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:27:07.549718 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bf896e-9923-4f8a-a4ce-f5252dfaf44e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:27:07.550313 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bf896e-9923-4f8a-a4ce-f5252dfaf44e None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:27:07.550590 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bf896e-9923-4f8a-a4ce-f5252dfaf44e None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:27:07.550857 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bf896e-9923-4f8a-a4ce-f5252dfaf44e None admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:27:07.551493 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bf896e-9923-4f8a-a4ce-f5252dfaf44e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:27:07.551794 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bf896e-9923-4f8a-a4ce-f5252dfaf44e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:27:07.553169 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bf896e-9923-4f8a-a4ce-f5252dfaf44e None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:27:07.567014 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 638/1284] 10.222.0.57 () {56 vars in 1109 bytes} [Wed Aug 10 08:27:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:27:07.572722 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6d86ae-0d1e-41ff-ba5c-d4581cf1df61 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:27:07.670802 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6d86ae-0d1e-41ff-ba5c-d4581cf1df61 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:27:07.684035 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:27:07.704318 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842f0029-de85-4eb6-b96e-e100eac8b346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:27:07.704877 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842f0029-de85-4eb6-b96e-e100eac8b346 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:27:07.705178 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842f0029-de85-4eb6-b96e-e100eac8b346 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:27:07.705445 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842f0029-de85-4eb6-b96e-e100eac8b346 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:27:07.711792 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842f0029-de85-4eb6-b96e-e100eac8b346 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:27:07.713283 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842f0029-de85-4eb6-b96e-e100eac8b346 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:27:07.724083 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application [None req-842f0029-de85-4eb6-b96e-e100eac8b346 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 08:27:07.724083 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:27:07.724083 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:27:07.724083 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:27:07.724083 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:27:07.724083 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:27:07.724083 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:27:07.724083 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:27:07.724083 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:27:07.724083 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:27:07.724083 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:27:07.724083 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:27:07.724083 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 08:27:07.724083 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 08:27:07.724083 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 08:27:07.726644 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 08:27:07.726644 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:27:07.726644 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:27:07.726644 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:27:07.726644 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:27:07.726644 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:27:07.726644 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:27:07.726644 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:27:07.726644 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application with Lock( Aug 10 08:27:07.726644 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:27:07.726644 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:27:07.726644 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:27:07.726644 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:27:07.726644 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:27:07.726644 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:27:07.726644 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:27:07.726644 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:27:07.727777 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 08:27:07.727777 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 08:27:07.727777 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 08:27:07.727777 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 08:27:07.727777 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 08:27:07.727777 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 08:27:07.727777 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 08:27:07.727777 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: ERROR keystone.server.flask.application Aug 10 08:27:07.727777 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 647/1285] 10.222.0.57 () {56 vars in 1111 bytes} [Wed Aug 10 08:27:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:27:07.731030 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bf896e-9923-4f8a-a4ce-f5252dfaf44e None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:27:07.782338 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:27:07.833203 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bf896e-9923-4f8a-a4ce-f5252dfaf44e None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:27:07.866679 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763d081d-ab16-4674-a75e-bcf7d329b93b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:27:07.867274 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763d081d-ab16-4674-a75e-bcf7d329b93b None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:27:07.867588 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763d081d-ab16-4674-a75e-bcf7d329b93b None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:27:07.867850 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763d081d-ab16-4674-a75e-bcf7d329b93b None admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:27:07.868796 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763d081d-ab16-4674-a75e-bcf7d329b93b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:27:07.869121 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763d081d-ab16-4674-a75e-bcf7d329b93b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:27:07.870593 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763d081d-ab16-4674-a75e-bcf7d329b93b None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:27:07.882652 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 639/1286] 10.222.0.57 () {56 vars in 1148 bytes} [Wed Aug 10 08:27:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:27:07.888810 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842f0029-de85-4eb6-b96e-e100eac8b346 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:27:07.989596 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842f0029-de85-4eb6-b96e-e100eac8b346 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:27:08.022496 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32980817-4a20-4dc0-89ff-fc717113f79d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:27:08.023087 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32980817-4a20-4dc0-89ff-fc717113f79d None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:27:08.023360 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32980817-4a20-4dc0-89ff-fc717113f79d None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:27:08.023622 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32980817-4a20-4dc0-89ff-fc717113f79d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:27:08.024326 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32980817-4a20-4dc0-89ff-fc717113f79d None admin] RBAC: Adding query filter params (role.id=ec3751692d9c4ecf81b4a71c936c46be, scope.project.id=b49e381419e940278bcb6111899589b5, user.id=2912307c56594125b157f1cfdf7c96a9) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:27:08.024624 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32980817-4a20-4dc0-89ff-fc717113f79d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:27:08.026064 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32980817-4a20-4dc0-89ff-fc717113f79d None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:27:08.031801 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 648/1287] 10.222.0.57 () {56 vars in 1374 bytes} [Wed Aug 10 08:27:07 2022] GET /identity/v3/role_assignments?user.id=2912307c56594125b157f1cfdf7c96a9&scope.project.id=b49e381419e940278bcb6111899589b5&role.id=ec3751692d9c4ecf81b4a71c936c46be => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:27:10.080316 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1d7ce1-ee46-43db-982a-99a51c84787d None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:27:10.080846 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1d7ce1-ee46-43db-982a-99a51c84787d None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:27:10.080846 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1d7ce1-ee46-43db-982a-99a51c84787d None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:27:10.082260 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 640/1288] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:27:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:27:10.087437 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bb49cb-73e0-4f4a-9ed2-4859f6785aff None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:27:10.087836 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bb49cb-73e0-4f4a-9ed2-4859f6785aff None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:27:10.087986 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bb49cb-73e0-4f4a-9ed2-4859f6785aff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:27:10.106121 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-53bb49cb-73e0-4f4a-9ed2-4859f6785aff None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:27:10.107313 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-53bb49cb-73e0-4f4a-9ed2-4859f6785aff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:27:10.136682 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:27:10.174900 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 649/1289] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:27:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 08:27:10.197799 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1d7ce1-ee46-43db-982a-99a51c84787d None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:27:10.298563 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1d7ce1-ee46-43db-982a-99a51c84787d None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:27:10.308972 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:27:10.332050 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdbc4bb-c812-4f58-94aa-b85604eaf0ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:27:10.332629 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdbc4bb-c812-4f58-94aa-b85604eaf0ca None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:27:10.332906 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdbc4bb-c812-4f58-94aa-b85604eaf0ca None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:27:10.333202 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdbc4bb-c812-4f58-94aa-b85604eaf0ca None admin] PATH_INFO: `/v3/services/placement` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:27:10.333831 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdbc4bb-c812-4f58-94aa-b85604eaf0ca None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:27:10.335202 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdbc4bb-c812-4f58-94aa-b85604eaf0ca None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:27:10.343651 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-2fdbc4bb-c812-4f58-94aa-b85604eaf0ca None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 08:27:10.343651 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:27:10.343651 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 08:27:10.343651 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 08:27:10.343651 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 08:27:10.343651 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 08:27:10.343651 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 08:27:10.343651 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 08:27:10.343651 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 08:27:10.343651 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:27:10.343651 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 08:27:10.343651 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:27:10.343651 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:27:10.343651 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:27:10.343651 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:27:10.343651 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:27:10.343651 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:27:10.344905 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:27:10.344905 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:27:10.344905 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:27:10.344905 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:27:10.344905 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:27:10.344905 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:27:10.344905 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 08:27:10.344905 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 08:27:10.344905 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 08:27:10.344905 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 08:27:10.344905 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:27:10.344905 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:27:10.344905 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:27:10.344905 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:27:10.344905 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:27:10.344905 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:27:10.346070 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:27:10.346070 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:27:10.346070 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:27:10.346070 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:27:10.346070 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:27:10.346070 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:27:10.346070 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:27:10.346070 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:27:10.346070 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:27:10.346070 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:27:10.346070 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 08:27:10.346070 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 08:27:10.346070 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 08:27:10.346070 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:27:10.346070 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 641/1290] 10.222.0.57 () {56 vars in 1117 bytes} [Wed Aug 10 08:27:10 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:27:10.350868 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bb49cb-73e0-4f4a-9ed2-4859f6785aff None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:27:10.450586 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bb49cb-73e0-4f4a-9ed2-4859f6785aff None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:27:10.484514 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d073325e-5c21-4a88-928a-3a08fd151d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:27:10.485403 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d073325e-5c21-4a88-928a-3a08fd151d63 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:27:10.485650 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d073325e-5c21-4a88-928a-3a08fd151d63 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:27:10.485923 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d073325e-5c21-4a88-928a-3a08fd151d63 None admin] PATH_INFO: `/v3/services` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:27:10.486555 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d073325e-5c21-4a88-928a-3a08fd151d63 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:27:10.486857 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d073325e-5c21-4a88-928a-3a08fd151d63 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:27:10.488222 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d073325e-5c21-4a88-928a-3a08fd151d63 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:27:10.493099 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 650/1291] 10.222.0.57 () {56 vars in 1116 bytes} [Wed Aug 10 08:27:10 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:27:10.499133 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdbc4bb-c812-4f58-94aa-b85604eaf0ca None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:27:10.598409 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdbc4bb-c812-4f58-94aa-b85604eaf0ca None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:27:10.631230 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fe6579-98fb-4e93-9e7e-53722f3c4c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:27:10.631940 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fe6579-98fb-4e93-9e7e-53722f3c4c86 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:27:10.632311 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fe6579-98fb-4e93-9e7e-53722f3c4c86 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:27:10.632670 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fe6579-98fb-4e93-9e7e-53722f3c4c86 None admin] PATH_INFO: `/v3/services` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:27:10.633469 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fe6579-98fb-4e93-9e7e-53722f3c4c86 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:27:10.633870 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fe6579-98fb-4e93-9e7e-53722f3c4c86 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:27:10.635336 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fe6579-98fb-4e93-9e7e-53722f3c4c86 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:27:10.639669 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 642/1292] 10.222.0.57 () {56 vars in 1116 bytes} [Wed Aug 10 08:27:10 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:27:12.650470 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3911c757-2ae4-499a-ad1f-f1e0c4c21830 None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:27:12.651048 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3911c757-2ae4-499a-ad1f-f1e0c4c21830 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:27:12.651048 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3911c757-2ae4-499a-ad1f-f1e0c4c21830 None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:27:12.653293 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 651/1293] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:27:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 08:27:12.658825 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3883e68-b0c4-4183-a7cf-8d8e498a2808 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:27:12.659334 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3883e68-b0c4-4183-a7cf-8d8e498a2808 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:27:12.659447 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3883e68-b0c4-4183-a7cf-8d8e498a2808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:27:12.678548 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-e3883e68-b0c4-4183-a7cf-8d8e498a2808 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:27:12.679726 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-e3883e68-b0c4-4183-a7cf-8d8e498a2808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:27:12.709837 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:27:12.751249 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 643/1294] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:27:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 08:27:12.776142 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3911c757-2ae4-499a-ad1f-f1e0c4c21830 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:27:12.811146 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:27:12.888407 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3911c757-2ae4-499a-ad1f-f1e0c4c21830 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:27:12.926298 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c98511e-5d97-4a99-bfbe-ff689a90f9cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:27:12.926908 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c98511e-5d97-4a99-bfbe-ff689a90f9cc None admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:27:12.927192 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c98511e-5d97-4a99-bfbe-ff689a90f9cc None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:27:12.927500 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c98511e-5d97-4a99-bfbe-ff689a90f9cc None admin] PATH_INFO: `/v3/services` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:27:12.928153 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c98511e-5d97-4a99-bfbe-ff689a90f9cc None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:27:12.929598 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c98511e-5d97-4a99-bfbe-ff689a90f9cc None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:27:13.092707 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 652/1295] 10.222.0.57 () {60 vars in 1141 bytes} [Wed Aug 10 08:27:12 2022] POST /identity/v3/services => generated 246 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:27:15.088758 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376466e2-7893-4292-96b4-75308bcc86e7 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:27:15.089328 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376466e2-7893-4292-96b4-75308bcc86e7 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:27:15.089328 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376466e2-7893-4292-96b4-75308bcc86e7 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:27:15.090703 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 644/1296] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:27:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:27:15.096335 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf393347-1445-4049-a93c-1a9f0c2d5d60 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:27:15.096782 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf393347-1445-4049-a93c-1a9f0c2d5d60 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:27:15.097025 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf393347-1445-4049-a93c-1a9f0c2d5d60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:27:15.115169 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-cf393347-1445-4049-a93c-1a9f0c2d5d60 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:27:15.116348 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-cf393347-1445-4049-a93c-1a9f0c2d5d60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:27:15.189893 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 653/1297] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:27:15 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 08:27:15.213291 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376466e2-7893-4292-96b4-75308bcc86e7 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:27:15.314978 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376466e2-7893-4292-96b4-75308bcc86e7 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:27:15.350044 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c26100c-07df-4a99-9e0c-7b8069d8080f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:27:15.350597 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c26100c-07df-4a99-9e0c-7b8069d8080f None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:27:15.350873 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c26100c-07df-4a99-9e0c-7b8069d8080f None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:27:15.351140 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c26100c-07df-4a99-9e0c-7b8069d8080f None admin] PATH_INFO: `/v3/services/placement` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:27:15.351717 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c26100c-07df-4a99-9e0c-7b8069d8080f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:27:15.353109 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c26100c-07df-4a99-9e0c-7b8069d8080f None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:27:15.361698 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-4c26100c-07df-4a99-9e0c-7b8069d8080f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 08:27:15.361698 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:27:15.361698 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 08:27:15.361698 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 08:27:15.361698 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 08:27:15.361698 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 08:27:15.361698 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 08:27:15.361698 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 08:27:15.361698 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 08:27:15.361698 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:27:15.361698 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 08:27:15.361698 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:27:15.361698 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:27:15.361698 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:27:15.361698 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:27:15.361698 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:27:15.361698 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:27:15.362879 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:27:15.362879 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:27:15.362879 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:27:15.362879 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:27:15.362879 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:27:15.362879 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:27:15.362879 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 08:27:15.362879 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 08:27:15.362879 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 08:27:15.362879 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 08:27:15.362879 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:27:15.362879 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:27:15.362879 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:27:15.362879 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:27:15.362879 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:27:15.362879 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:27:15.363899 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:27:15.363899 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:27:15.363899 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:27:15.363899 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:27:15.363899 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:27:15.363899 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:27:15.363899 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:27:15.363899 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:27:15.363899 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:27:15.363899 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:27:15.363899 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 08:27:15.363899 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 08:27:15.363899 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 08:27:15.363899 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:27:15.363899 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 645/1298] 10.222.0.57 () {56 vars in 1117 bytes} [Wed Aug 10 08:27:15 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 08:27:15.369125 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf393347-1445-4049-a93c-1a9f0c2d5d60 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:27:15.387654 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:27:15.468492 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf393347-1445-4049-a93c-1a9f0c2d5d60 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:27:15.504765 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d4dc26-bc7a-4b5c-8413-eb0a32d08c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:27:15.505407 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d4dc26-bc7a-4b5c-8413-eb0a32d08c96 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:27:15.505688 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d4dc26-bc7a-4b5c-8413-eb0a32d08c96 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:27:15.505959 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d4dc26-bc7a-4b5c-8413-eb0a32d08c96 None admin] PATH_INFO: `/v3/services` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:27:15.506593 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d4dc26-bc7a-4b5c-8413-eb0a32d08c96 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:27:15.506900 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d4dc26-bc7a-4b5c-8413-eb0a32d08c96 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:27:15.508306 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d4dc26-bc7a-4b5c-8413-eb0a32d08c96 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:27:15.513556 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 654/1299] 10.222.0.57 () {56 vars in 1116 bytes} [Wed Aug 10 08:27:15 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:27:15.520279 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c26100c-07df-4a99-9e0c-7b8069d8080f None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:27:15.552313 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:27:15.620179 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c26100c-07df-4a99-9e0c-7b8069d8080f None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:27:15.653464 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1b4280-0a40-4aa5-bf27-6e55d145bcaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:27:15.654049 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1b4280-0a40-4aa5-bf27-6e55d145bcaf None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:27:15.654326 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1b4280-0a40-4aa5-bf27-6e55d145bcaf None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:27:15.654591 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1b4280-0a40-4aa5-bf27-6e55d145bcaf None admin] PATH_INFO: `/v3/endpoints` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:27:15.655276 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1b4280-0a40-4aa5-bf27-6e55d145bcaf None admin] RBAC: Adding query filter params (interface=public, service_id=97fe6c64d5cf4623a0cf265d87e4bd67, region_id=RegionOne) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:27:15.655583 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1b4280-0a40-4aa5-bf27-6e55d145bcaf None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:27:15.658003 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1b4280-0a40-4aa5-bf27-6e55d145bcaf None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:27:15.663052 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 646/1300] 10.222.0.57 () {56 vars in 1251 bytes} [Wed Aug 10 08:27:15 2022] GET /identity/v3/endpoints?service_id=97fe6c64d5cf4623a0cf265d87e4bd67&interface=public®ion_id=RegionOne => generated 194 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:27:15.668527 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d4dc26-bc7a-4b5c-8413-eb0a32d08c96 None admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:27:15.747323 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:27:15.764725 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d4dc26-bc7a-4b5c-8413-eb0a32d08c96 None admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:27:15.799970 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e222325-e82d-4a8a-8014-efeadfeaae1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:27:15.800553 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e222325-e82d-4a8a-8014-efeadfeaae1a None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:27:15.800832 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e222325-e82d-4a8a-8014-efeadfeaae1a None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:27:15.801130 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e222325-e82d-4a8a-8014-efeadfeaae1a None admin] PATH_INFO: `/v3/services` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:27:15.801720 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e222325-e82d-4a8a-8014-efeadfeaae1a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:27:15.803189 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e222325-e82d-4a8a-8014-efeadfeaae1a None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:27:15.809850 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 655/1301] 10.222.0.57 () {56 vars in 1087 bytes} [Wed Aug 10 08:27:15 2022] GET /identity/v3/services => generated 2191 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:27:17.820614 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92700e9d-5081-4175-8062-5794ab756b39 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:27:17.821181 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92700e9d-5081-4175-8062-5794ab756b39 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:27:17.821181 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92700e9d-5081-4175-8062-5794ab756b39 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:27:17.822609 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 647/1302] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:27:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:27:17.827576 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cf9f3e-08bb-47bb-90f7-059706b22393 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:27:17.828022 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cf9f3e-08bb-47bb-90f7-059706b22393 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:27:17.828166 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cf9f3e-08bb-47bb-90f7-059706b22393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:27:17.845660 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-a5cf9f3e-08bb-47bb-90f7-059706b22393 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:27:17.846839 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-a5cf9f3e-08bb-47bb-90f7-059706b22393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:27:17.910979 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 656/1303] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:27:17 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 08:27:17.934065 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92700e9d-5081-4175-8062-5794ab756b39 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:27:18.028127 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:27:18.035224 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92700e9d-5081-4175-8062-5794ab756b39 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:27:18.068627 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e666ae-6657-4e70-8137-4e331176ad98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:27:18.069271 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e666ae-6657-4e70-8137-4e331176ad98 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:27:18.069552 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e666ae-6657-4e70-8137-4e331176ad98 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:27:18.069822 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e666ae-6657-4e70-8137-4e331176ad98 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:27:18.070401 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e666ae-6657-4e70-8137-4e331176ad98 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:27:18.071801 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e666ae-6657-4e70-8137-4e331176ad98 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:27:18.080492 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-d6e666ae-6657-4e70-8137-4e331176ad98 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 08:27:18.080492 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:27:18.080492 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 08:27:18.080492 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 08:27:18.080492 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 08:27:18.080492 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 08:27:18.080492 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 08:27:18.080492 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 08:27:18.080492 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 08:27:18.080492 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:27:18.080492 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 08:27:18.080492 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:27:18.080492 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 08:27:18.080492 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 08:27:18.080492 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 08:27:18.080492 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 08:27:18.080492 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 08:27:18.081769 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 08:27:18.081769 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 08:27:18.081769 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 08:27:18.081769 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 08:27:18.081769 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 08:27:18.081769 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 08:27:18.081769 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 08:27:18.081769 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 08:27:18.081769 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 08:27:18.081769 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 08:27:18.081769 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 08:27:18.081769 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 08:27:18.081769 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 08:27:18.081769 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 08:27:18.081769 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 08:27:18.081769 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 08:27:18.082634 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 08:27:18.082634 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Aug 10 08:27:18.082634 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 08:27:18.082634 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Aug 10 08:27:18.082634 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 08:27:18.082634 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 08:27:18.082634 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 08:27:18.082634 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Aug 10 08:27:18.082634 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 08:27:18.082634 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Aug 10 08:27:18.082634 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 08:27:18.082634 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 08:27:18.082634 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 08:27:18.082634 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Aug 10 08:27:18.082634 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 648/1304] 10.222.0.57 () {56 vars in 1117 bytes} [Wed Aug 10 08:27:17 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 08:27:18.087598 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cf9f3e-08bb-47bb-90f7-059706b22393 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:27:18.163216 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:27:18.185307 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cf9f3e-08bb-47bb-90f7-059706b22393 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:27:18.218672 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcb170c-6d5a-4392-bbe8-ab5715eda528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:27:18.219836 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcb170c-6d5a-4392-bbe8-ab5715eda528 None admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:27:18.219836 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcb170c-6d5a-4392-bbe8-ab5715eda528 None admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:27:18.220314 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcb170c-6d5a-4392-bbe8-ab5715eda528 None admin] PATH_INFO: `/v3/services` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:27:18.220906 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcb170c-6d5a-4392-bbe8-ab5715eda528 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:27:18.221232 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcb170c-6d5a-4392-bbe8-ab5715eda528 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:27:18.222593 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcb170c-6d5a-4392-bbe8-ab5715eda528 None admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:27:18.227637 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 657/1305] 10.222.0.57 () {56 vars in 1116 bytes} [Wed Aug 10 08:27:18 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:27:18.233759 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e666ae-6657-4e70-8137-4e331176ad98 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:27:18.332888 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e666ae-6657-4e70-8137-4e331176ad98 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:27:18.370065 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923815ff-d195-40cd-8c60-e56e5cbf6e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:27:18.370576 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923815ff-d195-40cd-8c60-e56e5cbf6e29 None admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:27:18.370849 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923815ff-d195-40cd-8c60-e56e5cbf6e29 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:27:18.371111 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923815ff-d195-40cd-8c60-e56e5cbf6e29 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:27:18.371753 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923815ff-d195-40cd-8c60-e56e5cbf6e29 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:27:18.373201 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923815ff-d195-40cd-8c60-e56e5cbf6e29 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:27:18.441045 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 649/1306] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 08:27:18 2022] POST /identity/v3/endpoints => generated 331 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:27:28.224753 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=30 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:27:28.435863 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:33:32.242821 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf177732-f200-4190-a3ca-8812769248ae None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:33:32.243265 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf177732-f200-4190-a3ca-8812769248ae None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:33:32.243361 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf177732-f200-4190-a3ca-8812769248ae None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:33:32.245115 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 658/1307] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:33:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:33:32.254478 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809a1eec-997d-46bf-994e-ee03f41dc392 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:33:32.254478 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809a1eec-997d-46bf-994e-ee03f41dc392 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:33:32.254478 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809a1eec-997d-46bf-994e-ee03f41dc392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:33:32.364166 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-809a1eec-997d-46bf-994e-ee03f41dc392 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:33:32.365925 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-809a1eec-997d-46bf-994e-ee03f41dc392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:33:32.504220 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 650/1308] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:33:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:33:32.548408 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b1c4c5-e220-4134-879b-03fce00116f5 None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:33:32.548689 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b1c4c5-e220-4134-879b-03fce00116f5 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:33:32.549189 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b1c4c5-e220-4134-879b-03fce00116f5 None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:33:32.550687 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 659/1309] 10.222.0.57 () {54 vars in 965 bytes} [Wed Aug 10 08:33:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:33:32.557785 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b91378-feb8-45f1-a144-1f3584011179 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:33:32.558306 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b91378-feb8-45f1-a144-1f3584011179 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:33:32.558737 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b91378-feb8-45f1-a144-1f3584011179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:33:32.643970 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-51b91378-feb8-45f1-a144-1f3584011179 None None] MFA Rules not processed for user `d4863ecff21e408c9deca645b88d3727`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:33:32.645154 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-51b91378-feb8-45f1-a144-1f3584011179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:33:32.671051 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:33:32.734401 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 651/1310] 10.222.0.57 () {58 vars in 1062 bytes} [Wed Aug 10 08:33:32 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 08:33:32.743364 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b1c4c5-e220-4134-879b-03fce00116f5 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:33:32.746103 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-a4b1c4c5-e220-4134-879b-03fce00116f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:33:32.858704 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b1c4c5-e220-4134-879b-03fce00116f5 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:33:32.937275 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7c88c9-231f-445d-a340-45c87037822c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:33:32.937776 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7c88c9-231f-445d-a340-45c87037822c service neutron] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:33:32.938043 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7c88c9-231f-445d-a340-45c87037822c service neutron] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:33:32.938316 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7c88c9-231f-445d-a340-45c87037822c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:33:32.939963 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-ad7c88c9-231f-445d-a340-45c87037822c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:33:33.003693 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:33:33.065168 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7c88c9-231f-445d-a340-45c87037822c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:33:33.066840 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7c88c9-231f-445d-a340-45c87037822c service neutron] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:33:33.090172 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 660/1311] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:33:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:33:35.433551 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d59697-a48c-4916-affb-74b4be3e9254 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:33:35.434172 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d59697-a48c-4916-affb-74b4be3e9254 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:33:35.434284 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d59697-a48c-4916-affb-74b4be3e9254 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:33:35.435739 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 652/1312] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:33:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:33:35.441423 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc56beb-e591-4863-800c-946765107f26 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:33:35.441948 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc56beb-e591-4863-800c-946765107f26 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:33:35.441948 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc56beb-e591-4863-800c-946765107f26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:33:35.519581 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-adc56beb-e591-4863-800c-946765107f26 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:33:35.520833 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-adc56beb-e591-4863-800c-946765107f26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:33:35.617256 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 661/1313] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:33:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:33:35.672356 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fba562-6b17-4e85-93f2-b5e22011dfff None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:33:35.672638 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fba562-6b17-4e85-93f2-b5e22011dfff None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:33:35.672932 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fba562-6b17-4e85-93f2-b5e22011dfff None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:33:35.674288 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 653/1314] 10.222.0.57 () {54 vars in 965 bytes} [Wed Aug 10 08:33:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:33:35.680469 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9fe4b1-4f58-4cb1-b72c-0126f0ec3ea7 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:33:35.680757 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9fe4b1-4f58-4cb1-b72c-0126f0ec3ea7 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:33:35.681066 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9fe4b1-4f58-4cb1-b72c-0126f0ec3ea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:33:35.693373 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:33:35.758549 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-eb9fe4b1-4f58-4cb1-b72c-0126f0ec3ea7 None None] MFA Rules not processed for user `d4863ecff21e408c9deca645b88d3727`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:33:35.759988 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-eb9fe4b1-4f58-4cb1-b72c-0126f0ec3ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:33:35.836161 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 662/1315] 10.222.0.57 () {58 vars in 1062 bytes} [Wed Aug 10 08:33:35 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:33:35.845694 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fba562-6b17-4e85-93f2-b5e22011dfff None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:33:35.847242 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-c1fba562-6b17-4e85-93f2-b5e22011dfff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:33:35.950741 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fba562-6b17-4e85-93f2-b5e22011dfff None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:33:36.019830 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:33:36.022131 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecf402a-af89-426b-a4b3-08b0fa271c40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:33:36.022728 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecf402a-af89-426b-a4b3-08b0fa271c40 service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:33:36.023004 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecf402a-af89-426b-a4b3-08b0fa271c40 service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:33:36.023270 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecf402a-af89-426b-a4b3-08b0fa271c40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:33:36.024853 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-6ecf402a-af89-426b-a4b3-08b0fa271c40 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:33:36.132450 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecf402a-af89-426b-a4b3-08b0fa271c40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:33:36.133923 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecf402a-af89-426b-a4b3-08b0fa271c40 service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:33:36.152285 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 654/1316] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:33:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:33:38.842667 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d3702a-6abf-45ef-bad6-03843241cea4 None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:33:38.843239 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d3702a-6abf-45ef-bad6-03843241cea4 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:33:38.843239 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d3702a-6abf-45ef-bad6-03843241cea4 None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:33:38.844572 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 663/1317] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:33:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:33:38.849748 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d773c2f-3585-47e9-8083-2b5f2d2617cf None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:33:38.850166 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d773c2f-3585-47e9-8083-2b5f2d2617cf None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:33:38.850278 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d773c2f-3585-47e9-8083-2b5f2d2617cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:33:38.916289 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-1d773c2f-3585-47e9-8083-2b5f2d2617cf None None] MFA Rules not processed for user `8179b7e759db449e9295e696b2333c9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:33:38.917617 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-1d773c2f-3585-47e9-8083-2b5f2d2617cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:33:38.969381 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:33:39.013490 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 655/1318] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:33:38 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:33:39.053195 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d3702a-6abf-45ef-bad6-03843241cea4 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:33:39.054755 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-02d3702a-6abf-45ef-bad6-03843241cea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:33:39.158809 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d3702a-6abf-45ef-bad6-03843241cea4 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:33:39.161567 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:33:39.231962 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7079d9d1-232e-4c2c-9ebf-0ae52ee315a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:33:39.232609 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7079d9d1-232e-4c2c-9ebf-0ae52ee315a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:33:39.232880 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7079d9d1-232e-4c2c-9ebf-0ae52ee315a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:33:39.233173 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7079d9d1-232e-4c2c-9ebf-0ae52ee315a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:33:39.237386 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-7079d9d1-232e-4c2c-9ebf-0ae52ee315a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:33:39.341599 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7079d9d1-232e-4c2c-9ebf-0ae52ee315a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:33:39.343023 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7079d9d1-232e-4c2c-9ebf-0ae52ee315a5 service neutron] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:33:39.361582 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 664/1319] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:33:39 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:33:42.386813 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacd27c5-ee4c-4df0-b34d-9dba920c4059 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:33:42.387384 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacd27c5-ee4c-4df0-b34d-9dba920c4059 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:33:42.387384 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacd27c5-ee4c-4df0-b34d-9dba920c4059 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:33:42.388755 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 656/1320] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:33:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:33:42.393924 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5330f9f9-015c-45f4-8aa4-f2ee509756fb None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:33:42.394356 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5330f9f9-015c-45f4-8aa4-f2ee509756fb None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:33:42.394464 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5330f9f9-015c-45f4-8aa4-f2ee509756fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:33:42.435097 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:33:42.459124 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-5330f9f9-015c-45f4-8aa4-f2ee509756fb None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:33:42.460316 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-5330f9f9-015c-45f4-8aa4-f2ee509756fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:33:42.554307 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 665/1321] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:33:42 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:33:42.594485 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacd27c5-ee4c-4df0-b34d-9dba920c4059 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:33:42.596045 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-bacd27c5-ee4c-4df0-b34d-9dba920c4059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:33:42.699470 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacd27c5-ee4c-4df0-b34d-9dba920c4059 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:33:42.770476 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41415118-985e-445c-9bf9-47b2a8f0b888 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:33:42.771068 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41415118-985e-445c-9bf9-47b2a8f0b888 service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:33:42.771341 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41415118-985e-445c-9bf9-47b2a8f0b888 service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:33:42.771605 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41415118-985e-445c-9bf9-47b2a8f0b888 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:33:42.773197 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-41415118-985e-445c-9bf9-47b2a8f0b888 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:33:42.780382 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:33:42.879610 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41415118-985e-445c-9bf9-47b2a8f0b888 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:33:42.881134 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41415118-985e-445c-9bf9-47b2a8f0b888 service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:33:42.899201 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 657/1322] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:33:42 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 08:33:45.896085 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24855eba-0e3f-43e1-9829-26d9fd52f86b None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:33:45.896616 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24855eba-0e3f-43e1-9829-26d9fd52f86b None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:33:45.896616 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24855eba-0e3f-43e1-9829-26d9fd52f86b None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:33:45.898005 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 666/1323] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:33:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:33:45.903464 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf24164-6b8e-4854-b55b-47d4cf093515 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:33:45.903861 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf24164-6b8e-4854-b55b-47d4cf093515 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:33:45.904015 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf24164-6b8e-4854-b55b-47d4cf093515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:33:45.970936 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-abf24164-6b8e-4854-b55b-47d4cf093515 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:33:45.972152 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-abf24164-6b8e-4854-b55b-47d4cf093515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:33:46.031696 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:33:46.073861 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 658/1324] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:33:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:33:46.111563 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24855eba-0e3f-43e1-9829-26d9fd52f86b None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:33:46.113133 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-24855eba-0e3f-43e1-9829-26d9fd52f86b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:33:46.227906 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24855eba-0e3f-43e1-9829-26d9fd52f86b None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:33:46.241927 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:33:46.308665 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90fe353-3c5e-45e6-8c76-c21c5a2c44ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:33:46.309223 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90fe353-3c5e-45e6-8c76-c21c5a2c44ea service neutron] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:33:46.309501 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90fe353-3c5e-45e6-8c76-c21c5a2c44ea service neutron] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:33:46.309771 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90fe353-3c5e-45e6-8c76-c21c5a2c44ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:33:46.311432 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-d90fe353-3c5e-45e6-8c76-c21c5a2c44ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:33:46.429889 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90fe353-3c5e-45e6-8c76-c21c5a2c44ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:33:46.431325 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90fe353-3c5e-45e6-8c76-c21c5a2c44ea service neutron] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:33:46.450680 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 667/1325] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:33:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:33:48.514524 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bf9fb9-0eb0-4b7a-8813-d8997722e96b None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:33:48.515081 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bf9fb9-0eb0-4b7a-8813-d8997722e96b None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:33:48.515081 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bf9fb9-0eb0-4b7a-8813-d8997722e96b None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:33:48.516417 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 659/1326] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:33:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:33:48.521614 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aa634d-fa2d-4f89-8c0d-235c8497112b None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:33:48.522043 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aa634d-fa2d-4f89-8c0d-235c8497112b None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:33:48.522169 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aa634d-fa2d-4f89-8c0d-235c8497112b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:33:48.570689 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:33:48.588821 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-26aa634d-fa2d-4f89-8c0d-235c8497112b None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:33:48.590866 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-26aa634d-fa2d-4f89-8c0d-235c8497112b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:33:48.687482 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 668/1327] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:33:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:33:48.723980 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bf9fb9-0eb0-4b7a-8813-d8997722e96b None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:33:48.725553 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-b4bf9fb9-0eb0-4b7a-8813-d8997722e96b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:33:48.830743 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bf9fb9-0eb0-4b7a-8813-d8997722e96b None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:33:48.906548 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f26a97-0d78-4bb7-93fc-e26fec25fa0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:33:48.907153 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f26a97-0d78-4bb7-93fc-e26fec25fa0a service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:33:48.907425 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f26a97-0d78-4bb7-93fc-e26fec25fa0a service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:33:48.907689 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f26a97-0d78-4bb7-93fc-e26fec25fa0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:33:48.909349 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-78f26a97-0d78-4bb7-93fc-e26fec25fa0a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:33:48.926498 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:33:49.018482 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f26a97-0d78-4bb7-93fc-e26fec25fa0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:33:49.019912 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f26a97-0d78-4bb7-93fc-e26fec25fa0a service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:33:49.039661 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 660/1328] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:33:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:33:51.123985 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74037675-9db8-4aae-857b-0f33838db59d None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:33:51.124493 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74037675-9db8-4aae-857b-0f33838db59d None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:33:51.124649 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74037675-9db8-4aae-857b-0f33838db59d None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:33:51.126101 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 669/1329] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:33:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:33:51.131777 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01162994-b379-4885-baad-69632ee695ca None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:33:51.132226 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01162994-b379-4885-baad-69632ee695ca None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:33:51.132446 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01162994-b379-4885-baad-69632ee695ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:33:51.202811 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-01162994-b379-4885-baad-69632ee695ca None None] MFA Rules not processed for user `8179b7e759db449e9295e696b2333c9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:33:51.204054 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-01162994-b379-4885-baad-69632ee695ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:33:51.271320 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:33:51.303757 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 661/1330] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:33:51 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:33:51.341419 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74037675-9db8-4aae-857b-0f33838db59d None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:33:51.342939 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-74037675-9db8-4aae-857b-0f33838db59d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:33:51.451412 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74037675-9db8-4aae-857b-0f33838db59d None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:33:51.473786 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:33:51.531389 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d98773-2c4e-4c6d-940d-188be28af5a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:33:51.532002 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d98773-2c4e-4c6d-940d-188be28af5a7 service neutron] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:33:51.532283 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d98773-2c4e-4c6d-940d-188be28af5a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:33:51.532748 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d98773-2c4e-4c6d-940d-188be28af5a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:33:51.534372 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-c3d98773-2c4e-4c6d-940d-188be28af5a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:33:51.648004 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d98773-2c4e-4c6d-940d-188be28af5a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:33:51.649539 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d98773-2c4e-4c6d-940d-188be28af5a7 service neutron] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:33:51.668296 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 670/1331] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:33:51 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:33:54.178502 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7a1207-9cd0-4504-9626-364df75a7b7f None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:33:54.179035 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7a1207-9cd0-4504-9626-364df75a7b7f None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:33:54.179035 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7a1207-9cd0-4504-9626-364df75a7b7f None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:33:54.180407 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 662/1332] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:33:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:33:54.186684 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa7b147-1254-45e1-8664-3ed80a8baf56 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:33:54.187093 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa7b147-1254-45e1-8664-3ed80a8baf56 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:33:54.187231 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa7b147-1254-45e1-8664-3ed80a8baf56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:33:54.250123 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:33:54.252838 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-7aa7b147-1254-45e1-8664-3ed80a8baf56 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:33:54.254088 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-7aa7b147-1254-45e1-8664-3ed80a8baf56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:33:54.349810 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 671/1333] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:33:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:33:54.387543 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7a1207-9cd0-4504-9626-364df75a7b7f None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:33:54.389337 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-ca7a1207-9cd0-4504-9626-364df75a7b7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:33:54.492454 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7a1207-9cd0-4504-9626-364df75a7b7f None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:33:54.565718 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c21ffb4-2a02-4e8b-8f23-e39be807c8b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:33:54.566252 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c21ffb4-2a02-4e8b-8f23-e39be807c8b3 service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:33:54.566529 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c21ffb4-2a02-4e8b-8f23-e39be807c8b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:33:54.566812 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c21ffb4-2a02-4e8b-8f23-e39be807c8b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:33:54.568431 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-8c21ffb4-2a02-4e8b-8f23-e39be807c8b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:33:54.594936 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:33:54.676274 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c21ffb4-2a02-4e8b-8f23-e39be807c8b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:33:54.677733 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c21ffb4-2a02-4e8b-8f23-e39be807c8b3 service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:33:54.696555 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 663/1334] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:33:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:33:56.762795 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed194739-4607-42cb-90ba-ce905bda4838 None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:33:56.763329 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed194739-4607-42cb-90ba-ce905bda4838 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:33:56.763329 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed194739-4607-42cb-90ba-ce905bda4838 None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:33:56.764731 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 672/1335] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:33:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:33:56.770187 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048c1443-f04d-401c-8c9e-1609c4c140e8 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:33:56.770622 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048c1443-f04d-401c-8c9e-1609c4c140e8 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:33:56.770724 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048c1443-f04d-401c-8c9e-1609c4c140e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:33:56.837175 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-048c1443-f04d-401c-8c9e-1609c4c140e8 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:33:56.838503 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-048c1443-f04d-401c-8c9e-1609c4c140e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:33:56.913230 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:33:56.936568 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 664/1336] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:33:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:33:56.976849 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed194739-4607-42cb-90ba-ce905bda4838 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:33:56.979167 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-ed194739-4607-42cb-90ba-ce905bda4838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:33:57.080953 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed194739-4607-42cb-90ba-ce905bda4838 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:33:57.108719 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:33:57.153366 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25211aa-4526-419b-bbed-bdb07e59efe1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:33:57.153971 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25211aa-4526-419b-bbed-bdb07e59efe1 service neutron] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:33:57.154252 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25211aa-4526-419b-bbed-bdb07e59efe1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:33:57.154523 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25211aa-4526-419b-bbed-bdb07e59efe1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:33:57.156108 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-c25211aa-4526-419b-bbed-bdb07e59efe1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:33:57.263781 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25211aa-4526-419b-bbed-bdb07e59efe1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:33:57.265242 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25211aa-4526-419b-bbed-bdb07e59efe1 service neutron] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:33:57.283567 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 673/1337] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:33:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:33:59.815042 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be127898-01b3-4900-b66b-9c09c9d35764 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:33:59.815543 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be127898-01b3-4900-b66b-9c09c9d35764 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:33:59.815642 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be127898-01b3-4900-b66b-9c09c9d35764 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:33:59.816954 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 665/1338] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:33:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:33:59.821977 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c46ab49-8181-41e0-b141-88953bb53277 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:33:59.822408 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c46ab49-8181-41e0-b141-88953bb53277 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:33:59.822594 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c46ab49-8181-41e0-b141-88953bb53277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:33:59.888416 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-6c46ab49-8181-41e0-b141-88953bb53277 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:33:59.889705 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-6c46ab49-8181-41e0-b141-88953bb53277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:33:59.898042 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:33:59.987609 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 674/1339] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:33:59 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:34:00.023845 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be127898-01b3-4900-b66b-9c09c9d35764 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:34:00.026096 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-be127898-01b3-4900-b66b-9c09c9d35764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:34:00.126253 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be127898-01b3-4900-b66b-9c09c9d35764 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:34:00.199302 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3993fe-c0d0-4cf8-b7de-59b1e25b1f53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:34:00.199900 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3993fe-c0d0-4cf8-b7de-59b1e25b1f53 service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:34:00.200177 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3993fe-c0d0-4cf8-b7de-59b1e25b1f53 service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:34:00.200447 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3993fe-c0d0-4cf8-b7de-59b1e25b1f53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:34:00.202069 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-6e3993fe-c0d0-4cf8-b7de-59b1e25b1f53 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:34:00.236751 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:34:00.308716 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3993fe-c0d0-4cf8-b7de-59b1e25b1f53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:34:00.310112 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3993fe-c0d0-4cf8-b7de-59b1e25b1f53 service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:34:00.328522 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 666/1340] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:34:00 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:34:04.641536 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b96d59-50fc-428e-b10a-95b2fdef9559 None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:34:04.642010 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b96d59-50fc-428e-b10a-95b2fdef9559 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:34:04.642097 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b96d59-50fc-428e-b10a-95b2fdef9559 None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:34:04.643420 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 675/1341] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:34:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:34:04.648611 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095aa1f4-23d4-4a54-8e5f-954629bcbd1b None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:34:04.649048 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095aa1f4-23d4-4a54-8e5f-954629bcbd1b None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:34:04.649182 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095aa1f4-23d4-4a54-8e5f-954629bcbd1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:34:04.716105 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-095aa1f4-23d4-4a54-8e5f-954629bcbd1b None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:34:04.717443 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-095aa1f4-23d4-4a54-8e5f-954629bcbd1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:34:04.805858 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:34:04.817041 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 667/1342] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:34:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:34:04.863127 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b96d59-50fc-428e-b10a-95b2fdef9559 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:34:04.863127 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-84b96d59-50fc-428e-b10a-95b2fdef9559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:34:04.964165 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b96d59-50fc-428e-b10a-95b2fdef9559 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:34:05.003611 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:34:05.039273 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445f62cf-0166-4874-b58b-ff6ac6143982 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:34:05.039898 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445f62cf-0166-4874-b58b-ff6ac6143982 service neutron] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:34:05.040462 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445f62cf-0166-4874-b58b-ff6ac6143982 service neutron] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:34:05.040973 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445f62cf-0166-4874-b58b-ff6ac6143982 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:34:05.042906 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-445f62cf-0166-4874-b58b-ff6ac6143982 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:34:05.161141 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445f62cf-0166-4874-b58b-ff6ac6143982 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:34:05.162584 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445f62cf-0166-4874-b58b-ff6ac6143982 service neutron] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:34:05.182852 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 676/1343] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:34:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:34:08.150144 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867b6e59-89ad-42cd-9f33-434feae227fb None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:34:08.150693 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867b6e59-89ad-42cd-9f33-434feae227fb None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:34:08.150693 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867b6e59-89ad-42cd-9f33-434feae227fb None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:34:08.151942 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 668/1344] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:34:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:34:08.157102 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1166a4a9-726c-4e2e-8bd8-329170a819ef None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:34:08.157525 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1166a4a9-726c-4e2e-8bd8-329170a819ef None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:34:08.157704 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1166a4a9-726c-4e2e-8bd8-329170a819ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:34:08.224520 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-1166a4a9-726c-4e2e-8bd8-329170a819ef None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:34:08.225741 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-1166a4a9-726c-4e2e-8bd8-329170a819ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:34:08.244779 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:34:08.323720 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 677/1345] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:34:08 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:34:08.361058 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867b6e59-89ad-42cd-9f33-434feae227fb None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:34:08.362598 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-867b6e59-89ad-42cd-9f33-434feae227fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:34:08.470009 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867b6e59-89ad-42cd-9f33-434feae227fb None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:34:08.546659 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e99bf0a-f3ca-4ff6-8c23-81a60538ece1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:34:08.547254 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e99bf0a-f3ca-4ff6-8c23-81a60538ece1 service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:34:08.547530 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e99bf0a-f3ca-4ff6-8c23-81a60538ece1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:34:08.547795 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e99bf0a-f3ca-4ff6-8c23-81a60538ece1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:34:08.549415 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-1e99bf0a-f3ca-4ff6-8c23-81a60538ece1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:34:08.594382 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:34:08.660853 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e99bf0a-f3ca-4ff6-8c23-81a60538ece1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:34:08.662324 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e99bf0a-f3ca-4ff6-8c23-81a60538ece1 service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:34:08.682013 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 669/1346] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:34:08 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:34:12.849980 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c61294f-e3fc-4753-bfaa-bc674e42c16d None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:34:12.850530 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c61294f-e3fc-4753-bfaa-bc674e42c16d None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:34:12.850530 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c61294f-e3fc-4753-bfaa-bc674e42c16d None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:34:12.855988 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 678/1347] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:34:12 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:34:12.857382 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343a5592-486a-4afe-aa4e-9af96ed94bbc None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:34:12.857833 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343a5592-486a-4afe-aa4e-9af96ed94bbc None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:34:12.857931 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343a5592-486a-4afe-aa4e-9af96ed94bbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:34:12.928632 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-343a5592-486a-4afe-aa4e-9af96ed94bbc None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:34:12.929875 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-343a5592-486a-4afe-aa4e-9af96ed94bbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:34:13.031655 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:34:13.034349 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 670/1348] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:34:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:34:13.073515 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c61294f-e3fc-4753-bfaa-bc674e42c16d None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:34:13.075087 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-9c61294f-e3fc-4753-bfaa-bc674e42c16d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:34:13.185238 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c61294f-e3fc-4753-bfaa-bc674e42c16d None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:34:13.236214 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:34:13.264792 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c070f91-7286-44d1-90ed-7bd8f4021099 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:34:13.265427 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c070f91-7286-44d1-90ed-7bd8f4021099 service neutron] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:34:13.265702 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c070f91-7286-44d1-90ed-7bd8f4021099 service neutron] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:34:13.265962 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c070f91-7286-44d1-90ed-7bd8f4021099 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:34:13.267596 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-0c070f91-7286-44d1-90ed-7bd8f4021099 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:34:13.380700 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c070f91-7286-44d1-90ed-7bd8f4021099 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:34:13.382174 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c070f91-7286-44d1-90ed-7bd8f4021099 service neutron] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:34:13.402624 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 679/1349] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:34:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:34:18.436573 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71cacf9-190a-4aa7-8e55-d41f13710c7c None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:34:18.437565 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71cacf9-190a-4aa7-8e55-d41f13710c7c None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:34:18.437987 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71cacf9-190a-4aa7-8e55-d41f13710c7c None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:34:18.439481 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 671/1350] 10.222.0.57 () {54 vars in 921 bytes} [Wed Aug 10 08:34:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:34:18.444898 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828a3be8-c3c1-41ee-882d-15de35a1509e None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:34:18.445434 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828a3be8-c3c1-41ee-882d-15de35a1509e None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:34:18.445515 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828a3be8-c3c1-41ee-882d-15de35a1509e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:34:18.539249 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-828a3be8-c3c1-41ee-882d-15de35a1509e None None] MFA Rules not processed for user `2912307c56594125b157f1cfdf7c96a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:34:18.541121 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-828a3be8-c3c1-41ee-882d-15de35a1509e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:34:18.579270 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:34:18.694378 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 680/1351] 10.222.0.57 () {58 vars in 1018 bytes} [Wed Aug 10 08:34:18 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:34:22.827792 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e031fe-5b61-4a17-812b-355f4e5abff2 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:34:22.828312 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e031fe-5b61-4a17-812b-355f4e5abff2 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:34:22.828312 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e031fe-5b61-4a17-812b-355f4e5abff2 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:34:22.829986 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 672/1352] 10.222.0.57 () {54 vars in 921 bytes} [Wed Aug 10 08:34:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:34:22.835599 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c49f2a7-6f7c-4aa9-a773-519192f69846 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:34:22.836051 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c49f2a7-6f7c-4aa9-a773-519192f69846 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:34:22.836158 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c49f2a7-6f7c-4aa9-a773-519192f69846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:34:22.916711 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-1c49f2a7-6f7c-4aa9-a773-519192f69846 None None] MFA Rules not processed for user `2912307c56594125b157f1cfdf7c96a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:34:22.918057 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-1c49f2a7-6f7c-4aa9-a773-519192f69846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:34:22.992076 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:34:23.029629 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 681/1353] 10.222.0.57 () {58 vars in 1018 bytes} [Wed Aug 10 08:34:22 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:34:24.345150 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe4d08c-11a0-450e-b56e-292bace7163b None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:34:24.345703 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe4d08c-11a0-450e-b56e-292bace7163b None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:34:24.345703 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe4d08c-11a0-450e-b56e-292bace7163b None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:34:24.347040 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 673/1354] 10.222.0.57 () {54 vars in 921 bytes} [Wed Aug 10 08:34:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:34:24.352183 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817430b3-a447-445f-89df-c04a1331b9ae None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:34:24.352647 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817430b3-a447-445f-89df-c04a1331b9ae None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:34:24.352745 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817430b3-a447-445f-89df-c04a1331b9ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:34:24.435662 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-817430b3-a447-445f-89df-c04a1331b9ae None None] MFA Rules not processed for user `2912307c56594125b157f1cfdf7c96a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:34:24.436940 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-817430b3-a447-445f-89df-c04a1331b9ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:34:24.542729 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 682/1355] 10.222.0.57 () {58 vars in 1018 bytes} [Wed Aug 10 08:34:24 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:34:28.077470 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ad375b-4725-495a-944f-cd7239aec152 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:34:28.078440 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ad375b-4725-495a-944f-cd7239aec152 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:34:28.078779 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ad375b-4725-495a-944f-cd7239aec152 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:34:28.080407 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 674/1356] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:34:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:34:28.086270 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fce6e1a-7720-463b-9c57-ef34c37182de None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:34:28.087330 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fce6e1a-7720-463b-9c57-ef34c37182de None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:34:28.088078 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fce6e1a-7720-463b-9c57-ef34c37182de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:34:28.128123 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:34:28.162435 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-2fce6e1a-7720-463b-9c57-ef34c37182de None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:34:28.163640 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-2fce6e1a-7720-463b-9c57-ef34c37182de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:34:28.268455 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 683/1357] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:34:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:34:28.326877 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cfc8f4-9fe9-4a51-bd8c-f03adf86aac5 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:34:28.327393 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cfc8f4-9fe9-4a51-bd8c-f03adf86aac5 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:34:28.327888 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cfc8f4-9fe9-4a51-bd8c-f03adf86aac5 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:34:28.330367 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 675/1358] 10.222.0.57 () {54 vars in 962 bytes} [Wed Aug 10 08:34:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:34:28.341120 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2423d99c-f8f1-4698-8280-9f14fe484262 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:34:28.341523 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2423d99c-f8f1-4698-8280-9f14fe484262 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:34:28.341975 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2423d99c-f8f1-4698-8280-9f14fe484262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:34:28.425787 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-2423d99c-f8f1-4698-8280-9f14fe484262 None None] MFA Rules not processed for user `a943008193914f76a663d489bd73e873`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:34:28.427007 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-2423d99c-f8f1-4698-8280-9f14fe484262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:34:28.488398 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:34:28.554547 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 684/1359] 10.222.0.57 () {58 vars in 1059 bytes} [Wed Aug 10 08:34:28 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:34:28.563033 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cfc8f4-9fe9-4a51-bd8c-f03adf86aac5 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:34:28.564539 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-95cfc8f4-9fe9-4a51-bd8c-f03adf86aac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:34:28.729350 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cfc8f4-9fe9-4a51-bd8c-f03adf86aac5 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:34:28.808838 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:34:28.853048 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1669bdd6-d3e9-436a-ad20-31eb1a7d9c07 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a943008193914f76a663d489bd73e873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:34:28.853653 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1669bdd6-d3e9-436a-ad20-31eb1a7d9c07 service nova] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:34:28.853935 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1669bdd6-d3e9-436a-ad20-31eb1a7d9c07 service nova] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:34:28.854206 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1669bdd6-d3e9-436a-ad20-31eb1a7d9c07 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:34:28.855830 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-1669bdd6-d3e9-436a-ad20-31eb1a7d9c07 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:34:28.952054 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1669bdd6-d3e9-436a-ad20-31eb1a7d9c07 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:34:28.954874 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1669bdd6-d3e9-436a-ad20-31eb1a7d9c07 service nova] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:34:28.975140 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 676/1360] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:34:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:34:31.250038 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa1336c-9d2b-4e39-9be4-9241d0294e32 None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:34:31.250592 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa1336c-9d2b-4e39-9be4-9241d0294e32 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:34:31.250592 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa1336c-9d2b-4e39-9be4-9241d0294e32 None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:34:31.251944 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 685/1361] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:34:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:34:31.257891 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c206d51f-4006-4c21-b7d2-1d50cf4c02c9 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:34:31.259555 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c206d51f-4006-4c21-b7d2-1d50cf4c02c9 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:34:31.260092 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c206d51f-4006-4c21-b7d2-1d50cf4c02c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:34:31.345231 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-c206d51f-4006-4c21-b7d2-1d50cf4c02c9 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:34:31.346842 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-c206d51f-4006-4c21-b7d2-1d50cf4c02c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:34:31.390892 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:34:31.463022 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 677/1362] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:34:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:34:31.510163 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa1336c-9d2b-4e39-9be4-9241d0294e32 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:34:31.511787 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-baa1336c-9d2b-4e39-9be4-9241d0294e32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:34:31.710754 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa1336c-9d2b-4e39-9be4-9241d0294e32 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:34:31.722217 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:34:31.835046 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8d9903-c3b0-4fcf-80dd-fa57028a4f58 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a943008193914f76a663d489bd73e873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:34:31.835736 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8d9903-c3b0-4fcf-80dd-fa57028a4f58 service nova] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:34:31.836106 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8d9903-c3b0-4fcf-80dd-fa57028a4f58 service nova] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:34:31.836585 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8d9903-c3b0-4fcf-80dd-fa57028a4f58 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:34:31.838315 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-1b8d9903-c3b0-4fcf-80dd-fa57028a4f58 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:34:31.939178 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8d9903-c3b0-4fcf-80dd-fa57028a4f58 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:34:31.940776 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8d9903-c3b0-4fcf-80dd-fa57028a4f58 service nova] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:34:31.960039 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 686/1363] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:34:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:34:34.527710 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65146c17-e6b5-49d5-9f9b-5dd369db401f None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:34:34.528239 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65146c17-e6b5-49d5-9f9b-5dd369db401f None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:34:34.528239 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65146c17-e6b5-49d5-9f9b-5dd369db401f None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:34:34.529703 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 678/1364] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:34:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:34:34.537246 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28268c0-33d2-4a88-a40d-e5454e305c77 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:34:34.537778 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28268c0-33d2-4a88-a40d-e5454e305c77 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:34:34.538129 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28268c0-33d2-4a88-a40d-e5454e305c77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:34:34.556431 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:34:34.637271 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-b28268c0-33d2-4a88-a40d-e5454e305c77 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:34:34.639698 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-b28268c0-33d2-4a88-a40d-e5454e305c77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:34:34.757561 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 687/1365] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:34:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:34:34.804613 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7624d2d-569f-42ad-9107-4e62e5597345 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:34:34.805586 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7624d2d-569f-42ad-9107-4e62e5597345 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:34:34.805958 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7624d2d-569f-42ad-9107-4e62e5597345 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:34:34.807684 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 679/1366] 10.222.0.57 () {54 vars in 962 bytes} [Wed Aug 10 08:34:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:34:34.814036 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a333ac29-8f27-4c63-8e3e-c9e46ed658b5 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:34:34.814382 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a333ac29-8f27-4c63-8e3e-c9e46ed658b5 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:34:34.814781 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a333ac29-8f27-4c63-8e3e-c9e46ed658b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:34:34.902961 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-a333ac29-8f27-4c63-8e3e-c9e46ed658b5 None None] MFA Rules not processed for user `a943008193914f76a663d489bd73e873`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:34:34.904208 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-a333ac29-8f27-4c63-8e3e-c9e46ed658b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:34:34.944021 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:34:35.043295 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 688/1367] 10.222.0.57 () {58 vars in 1059 bytes} [Wed Aug 10 08:34:34 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:34:35.044420 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7624d2d-569f-42ad-9107-4e62e5597345 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:34:35.046305 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-d7624d2d-569f-42ad-9107-4e62e5597345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:34:35.208885 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7624d2d-569f-42ad-9107-4e62e5597345 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:34:35.231733 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:34:35.340348 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53be60f2-749a-4c1a-8c4d-adbea956ff96 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a943008193914f76a663d489bd73e873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:34:35.340869 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53be60f2-749a-4c1a-8c4d-adbea956ff96 service nova] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:34:35.341248 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53be60f2-749a-4c1a-8c4d-adbea956ff96 service nova] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:34:35.342874 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53be60f2-749a-4c1a-8c4d-adbea956ff96 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:34:35.343564 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-53be60f2-749a-4c1a-8c4d-adbea956ff96 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:34:35.450784 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53be60f2-749a-4c1a-8c4d-adbea956ff96 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:34:35.452410 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53be60f2-749a-4c1a-8c4d-adbea956ff96 service nova] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:34:35.471976 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 680/1368] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:34:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:34:36.293596 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17f1fdc-bee1-4753-b50b-cff3010062a2 None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:34:36.294261 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17f1fdc-bee1-4753-b50b-cff3010062a2 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:34:36.294261 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17f1fdc-bee1-4753-b50b-cff3010062a2 None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:34:36.295512 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 689/1369] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:34:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:34:36.300515 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cacba79-27d1-49f5-84fd-203b9e683f08 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:34:36.300979 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cacba79-27d1-49f5-84fd-203b9e683f08 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:34:36.301079 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cacba79-27d1-49f5-84fd-203b9e683f08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:34:36.334068 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:34:36.370492 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-8cacba79-27d1-49f5-84fd-203b9e683f08 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:34:36.371724 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-8cacba79-27d1-49f5-84fd-203b9e683f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:34:36.477430 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 681/1370] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:34:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:34:36.822123 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ac4bac-c480-4207-a61a-a677b5b8fe7d None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:34:36.822123 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ac4bac-c480-4207-a61a-a677b5b8fe7d None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:34:36.822123 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ac4bac-c480-4207-a61a-a677b5b8fe7d None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:34:36.826382 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 690/1371] 10.222.0.57 () {54 vars in 964 bytes} [Wed Aug 10 08:34:36 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:34:36.831531 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2014f9b9-8073-4553-890f-3d35b43c3c28 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:34:36.831891 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2014f9b9-8073-4553-890f-3d35b43c3c28 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:34:36.832280 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2014f9b9-8073-4553-890f-3d35b43c3c28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:34:36.921631 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-2014f9b9-8073-4553-890f-3d35b43c3c28 None None] MFA Rules not processed for user `f1ba74e28ced4cbba857f54222dfbe41`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:34:36.922864 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-2014f9b9-8073-4553-890f-3d35b43c3c28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:34:36.972345 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:34:37.003516 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 682/1372] 10.222.0.57 () {58 vars in 1061 bytes} [Wed Aug 10 08:34:36 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:34:37.013348 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ac4bac-c480-4207-a61a-a677b5b8fe7d None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:34:37.014984 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-59ac4bac-c480-4207-a61a-a677b5b8fe7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:34:37.132756 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ac4bac-c480-4207-a61a-a677b5b8fe7d None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:34:37.175363 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:34:37.212546 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598e265d-0012-4001-aa1d-b1061afab63f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ba74e28ced4cbba857f54222dfbe41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:34:37.213185 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598e265d-0012-4001-aa1d-b1061afab63f service cinder] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:34:37.213493 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598e265d-0012-4001-aa1d-b1061afab63f service cinder] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:34:37.213732 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598e265d-0012-4001-aa1d-b1061afab63f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:34:37.215366 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-598e265d-0012-4001-aa1d-b1061afab63f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:34:37.331820 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598e265d-0012-4001-aa1d-b1061afab63f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:34:37.333452 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598e265d-0012-4001-aa1d-b1061afab63f service cinder] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:34:37.354384 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 691/1373] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:34:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:34:37.781392 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fb17bc-076d-41b3-a973-09a7b8bd5eda None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:34:37.781956 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fb17bc-076d-41b3-a973-09a7b8bd5eda None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:34:37.782302 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fb17bc-076d-41b3-a973-09a7b8bd5eda None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:34:37.783946 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 683/1374] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:34:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:34:37.790345 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea66fa88-f787-453a-b75d-ba6148171bf5 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:34:37.790633 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea66fa88-f787-453a-b75d-ba6148171bf5 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:34:37.790915 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea66fa88-f787-453a-b75d-ba6148171bf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:34:37.857852 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-ea66fa88-f787-453a-b75d-ba6148171bf5 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:34:37.858964 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-ea66fa88-f787-453a-b75d-ba6148171bf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:34:37.879418 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:34:37.957396 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 692/1375] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:34:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:34:38.003060 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fb17bc-076d-41b3-a973-09a7b8bd5eda None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:34:38.004692 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-23fb17bc-076d-41b3-a973-09a7b8bd5eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:34:38.155688 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fb17bc-076d-41b3-a973-09a7b8bd5eda None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:34:38.207797 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:34:38.270927 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a16f8e-85ef-4e32-97b4-7060486cfb20 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a943008193914f76a663d489bd73e873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:34:38.271534 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a16f8e-85ef-4e32-97b4-7060486cfb20 service nova] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:34:38.271810 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a16f8e-85ef-4e32-97b4-7060486cfb20 service nova] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:34:38.272079 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a16f8e-85ef-4e32-97b4-7060486cfb20 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:34:38.274322 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-82a16f8e-85ef-4e32-97b4-7060486cfb20 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:34:38.362889 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a16f8e-85ef-4e32-97b4-7060486cfb20 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:34:38.364402 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a16f8e-85ef-4e32-97b4-7060486cfb20 service nova] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:34:38.382529 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 684/1376] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:34:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:34:40.549418 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb762a3-7fc8-4564-90f4-9275990a62fc None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:34:40.549905 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb762a3-7fc8-4564-90f4-9275990a62fc None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:34:40.549905 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb762a3-7fc8-4564-90f4-9275990a62fc None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:34:40.551229 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 693/1377] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:34:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:34:40.556507 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8486d11-c398-4c35-866b-098758af9975 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:34:40.556990 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8486d11-c398-4c35-866b-098758af9975 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:34:40.557088 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8486d11-c398-4c35-866b-098758af9975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:34:40.622560 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-d8486d11-c398-4c35-866b-098758af9975 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:34:40.623749 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-d8486d11-c398-4c35-866b-098758af9975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:34:40.631096 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:34:40.722039 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 685/1378] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:34:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:34:40.766744 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb762a3-7fc8-4564-90f4-9275990a62fc None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:34:40.768312 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-8eb762a3-7fc8-4564-90f4-9275990a62fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:34:40.930337 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb762a3-7fc8-4564-90f4-9275990a62fc None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:34:40.933975 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:34:41.095172 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d9a314-ce6d-40d0-a072-c673727421ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a943008193914f76a663d489bd73e873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:34:41.095679 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d9a314-ce6d-40d0-a072-c673727421ea service nova] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:34:41.095958 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d9a314-ce6d-40d0-a072-c673727421ea service nova] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:34:41.096219 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d9a314-ce6d-40d0-a072-c673727421ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:34:41.098393 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-10d9a314-ce6d-40d0-a072-c673727421ea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:34:41.196221 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d9a314-ce6d-40d0-a072-c673727421ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:34:41.197874 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d9a314-ce6d-40d0-a072-c673727421ea service nova] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:34:41.217721 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 694/1379] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:34:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:34:43.431615 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb581b4-804b-4465-b01f-206acade6473 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:34:43.432169 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb581b4-804b-4465-b01f-206acade6473 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:34:43.432169 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb581b4-804b-4465-b01f-206acade6473 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:34:43.433715 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 686/1380] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:34:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:34:43.438630 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb5ae6c-943e-4ac9-abf3-74c1d8da8aef None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:34:43.439050 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb5ae6c-943e-4ac9-abf3-74c1d8da8aef None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:34:43.439250 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb5ae6c-943e-4ac9-abf3-74c1d8da8aef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:34:43.451745 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:34:43.515686 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-bdb5ae6c-943e-4ac9-abf3-74c1d8da8aef None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:34:43.516792 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-bdb5ae6c-943e-4ac9-abf3-74c1d8da8aef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:34:43.621532 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 695/1381] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:34:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:34:43.668128 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb581b4-804b-4465-b01f-206acade6473 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:34:43.669837 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-feb581b4-804b-4465-b01f-206acade6473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:34:43.835073 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:34:43.848529 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb581b4-804b-4465-b01f-206acade6473 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:34:43.984355 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8da58a3-ae58-4f52-85b5-0b829890fe0d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a943008193914f76a663d489bd73e873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:34:43.985123 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8da58a3-ae58-4f52-85b5-0b829890fe0d service nova] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:34:43.985651 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8da58a3-ae58-4f52-85b5-0b829890fe0d service nova] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:34:43.985928 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8da58a3-ae58-4f52-85b5-0b829890fe0d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:34:43.987821 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-d8da58a3-ae58-4f52-85b5-0b829890fe0d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:34:44.090564 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8da58a3-ae58-4f52-85b5-0b829890fe0d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:34:44.092067 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8da58a3-ae58-4f52-85b5-0b829890fe0d service nova] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:34:44.103456 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:34:44.112330 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 687/1382] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:34:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:34:46.336566 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b231d6-4e26-494a-ba63-309e12e6bd12 None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:34:46.337168 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b231d6-4e26-494a-ba63-309e12e6bd12 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:34:46.337168 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b231d6-4e26-494a-ba63-309e12e6bd12 None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:34:46.338575 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 696/1383] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:34:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:34:46.343901 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7927857-f77e-40ff-9b40-8aa80b699c21 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:34:46.344345 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7927857-f77e-40ff-9b40-8aa80b699c21 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:34:46.344455 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7927857-f77e-40ff-9b40-8aa80b699c21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:34:46.416411 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-c7927857-f77e-40ff-9b40-8aa80b699c21 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:34:46.418022 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-c7927857-f77e-40ff-9b40-8aa80b699c21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:34:46.522734 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 688/1384] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:34:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:34:46.565095 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b231d6-4e26-494a-ba63-309e12e6bd12 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:34:46.567190 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-29b231d6-4e26-494a-ba63-309e12e6bd12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:34:46.667545 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:34:46.732966 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b231d6-4e26-494a-ba63-309e12e6bd12 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:34:46.848279 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e590ef-3ba1-43d4-a929-73be5769550c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a943008193914f76a663d489bd73e873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:34:46.848892 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e590ef-3ba1-43d4-a929-73be5769550c service nova] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:34:46.849197 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e590ef-3ba1-43d4-a929-73be5769550c service nova] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:34:46.849478 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e590ef-3ba1-43d4-a929-73be5769550c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:34:46.851059 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-34e590ef-3ba1-43d4-a929-73be5769550c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:34:46.902694 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:34:46.945900 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e590ef-3ba1-43d4-a929-73be5769550c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:34:46.947476 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e590ef-3ba1-43d4-a929-73be5769550c service nova] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:34:46.967132 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 697/1385] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:34:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:34:49.345776 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b9d620-e82a-4dfc-9315-6879d28d7648 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:34:49.346339 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b9d620-e82a-4dfc-9315-6879d28d7648 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:34:49.346339 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b9d620-e82a-4dfc-9315-6879d28d7648 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:34:49.347697 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 689/1386] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:34:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:34:49.354070 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b133b6-5537-457f-8a88-75307949a5ff None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:34:49.354580 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b133b6-5537-457f-8a88-75307949a5ff None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:34:49.354753 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b133b6-5537-457f-8a88-75307949a5ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:34:49.424058 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-17b133b6-5537-457f-8a88-75307949a5ff None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:34:49.425359 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-17b133b6-5537-457f-8a88-75307949a5ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:34:49.530326 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 698/1387] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:34:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:34:49.578932 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b9d620-e82a-4dfc-9315-6879d28d7648 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:34:49.580689 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-e2b9d620-e82a-4dfc-9315-6879d28d7648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:34:49.673915 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:34:49.761846 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b9d620-e82a-4dfc-9315-6879d28d7648 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:34:49.891367 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e539b4-50c1-4540-b5ab-567a5c61ccd5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a943008193914f76a663d489bd73e873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:34:49.891969 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e539b4-50c1-4540-b5ab-567a5c61ccd5 service nova] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:34:49.892246 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e539b4-50c1-4540-b5ab-567a5c61ccd5 service nova] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:34:49.892515 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e539b4-50c1-4540-b5ab-567a5c61ccd5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:34:49.894147 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-39e539b4-50c1-4540-b5ab-567a5c61ccd5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:34:49.935347 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:34:49.995103 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e539b4-50c1-4540-b5ab-567a5c61ccd5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:34:49.996703 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e539b4-50c1-4540-b5ab-567a5c61ccd5 service nova] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:34:50.015958 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 690/1388] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:34:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:34:52.200618 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1409707c-5c39-4bb3-bb97-41b75464a0e4 None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:34:52.201224 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1409707c-5c39-4bb3-bb97-41b75464a0e4 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:34:52.201224 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1409707c-5c39-4bb3-bb97-41b75464a0e4 None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:34:52.202570 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 699/1389] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:34:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:34:52.207841 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2f4144-dd70-4f21-8c3b-8c641dbdaf84 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:34:52.208300 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2f4144-dd70-4f21-8c3b-8c641dbdaf84 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:34:52.208411 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2f4144-dd70-4f21-8c3b-8c641dbdaf84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:34:52.277048 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-7f2f4144-dd70-4f21-8c3b-8c641dbdaf84 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:34:52.278265 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-7f2f4144-dd70-4f21-8c3b-8c641dbdaf84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:34:52.378490 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 691/1390] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:34:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 08:34:52.423463 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1409707c-5c39-4bb3-bb97-41b75464a0e4 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:34:52.425183 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-1409707c-5c39-4bb3-bb97-41b75464a0e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:34:52.456400 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:34:52.573455 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1409707c-5c39-4bb3-bb97-41b75464a0e4 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:34:52.658594 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1cfbfe-3c2d-4578-a1ce-30631f6d9ff4 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:34:52.658883 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1cfbfe-3c2d-4578-a1ce-30631f6d9ff4 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:34:52.659151 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1cfbfe-3c2d-4578-a1ce-30631f6d9ff4 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:34:52.660538 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 692/1391] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:34:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 08:34:52.666617 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf721a2a-749c-4a3c-87e6-b5a8ea382044 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:34:52.666919 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf721a2a-749c-4a3c-87e6-b5a8ea382044 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:34:52.667190 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf721a2a-749c-4a3c-87e6-b5a8ea382044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:34:52.674854 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:34:52.683306 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:34:52.686907 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c018ba6e-e675-4ff9-b388-1a35b541363c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a943008193914f76a663d489bd73e873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:34:52.687514 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c018ba6e-e675-4ff9-b388-1a35b541363c service nova] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:34:52.687799 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c018ba6e-e675-4ff9-b388-1a35b541363c service nova] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:34:52.688067 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c018ba6e-e675-4ff9-b388-1a35b541363c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:34:52.689749 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-c018ba6e-e675-4ff9-b388-1a35b541363c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:34:52.737227 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-bf721a2a-749c-4a3c-87e6-b5a8ea382044 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:34:52.738509 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-bf721a2a-749c-4a3c-87e6-b5a8ea382044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:34:52.781440 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c018ba6e-e675-4ff9-b388-1a35b541363c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:34:52.783064 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c018ba6e-e675-4ff9-b388-1a35b541363c service nova] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:34:52.801846 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 700/1392] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:34:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:34:52.837950 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 693/1393] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:34:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:34:52.895752 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65b04ab-6ee8-41c6-b8e4-a6f8a979b21d None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:34:52.896041 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65b04ab-6ee8-41c6-b8e4-a6f8a979b21d None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:34:52.896318 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65b04ab-6ee8-41c6-b8e4-a6f8a979b21d None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:34:52.897697 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 701/1394] 10.222.0.57 () {54 vars in 967 bytes} [Wed Aug 10 08:34:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:34:52.902784 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daec089-ffb2-4341-b43e-9ec30071dce6 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:34:52.910518 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daec089-ffb2-4341-b43e-9ec30071dce6 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:34:52.910518 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daec089-ffb2-4341-b43e-9ec30071dce6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:34:52.981212 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-5daec089-ffb2-4341-b43e-9ec30071dce6 None None] MFA Rules not processed for user `3263af99fc2f48e79fd1e5b8a1e555a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:34:52.982461 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-5daec089-ffb2-4341-b43e-9ec30071dce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:34:53.022962 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:34:53.071023 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 694/1395] 10.222.0.57 () {58 vars in 1064 bytes} [Wed Aug 10 08:34:52 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 08:34:53.078621 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65b04ab-6ee8-41c6-b8e4-a6f8a979b21d None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:34:53.080199 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-c65b04ab-6ee8-41c6-b8e4-a6f8a979b21d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:34:53.190848 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65b04ab-6ee8-41c6-b8e4-a6f8a979b21d None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:34:53.205072 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:34:53.266081 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5ec713-cfa9-44c7-87dc-8a3e49558924 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3263af99fc2f48e79fd1e5b8a1e555a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:34:53.266681 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5ec713-cfa9-44c7-87dc-8a3e49558924 service glance] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:34:53.266956 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5ec713-cfa9-44c7-87dc-8a3e49558924 service glance] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:34:53.267220 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5ec713-cfa9-44c7-87dc-8a3e49558924 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:34:53.268793 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-2b5ec713-cfa9-44c7-87dc-8a3e49558924 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:34:53.381331 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5ec713-cfa9-44c7-87dc-8a3e49558924 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:34:53.383098 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5ec713-cfa9-44c7-87dc-8a3e49558924 service glance] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:34:53.403076 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 702/1396] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:34:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:34:54.262776 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9506a6-cdd3-4f42-bd00-68ea2ecc7891 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:34:54.264893 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9506a6-cdd3-4f42-bd00-68ea2ecc7891 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:34:54.264893 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9506a6-cdd3-4f42-bd00-68ea2ecc7891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:34:54.334527 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-fc9506a6-cdd3-4f42-bd00-68ea2ecc7891 None None] MFA Rules not processed for user `c140d31bd2b84c48a170e3ca90a041ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:34:54.335935 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-fc9506a6-cdd3-4f42-bd00-68ea2ecc7891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:34:54.427528 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 695/1397] 10.222.0.57 () {58 vars in 1009 bytes} [Wed Aug 10 08:34:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:34:54.475002 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b926e961-1665-46e6-b8fe-04459199265d None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:34:54.475547 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b926e961-1665-46e6-b8fe-04459199265d None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:34:54.475547 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b926e961-1665-46e6-b8fe-04459199265d None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:34:54.477169 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 703/1398] 10.222.0.57 () {54 vars in 943 bytes} [Wed Aug 10 08:34:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:34:54.483497 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7b4f17-d342-4fe7-ae8c-dbcee8ff1dbc None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:34:54.483726 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7b4f17-d342-4fe7-ae8c-dbcee8ff1dbc None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:34:54.484001 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7b4f17-d342-4fe7-ae8c-dbcee8ff1dbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:34:54.541357 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:34:54.560620 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-1f7b4f17-d342-4fe7-ae8c-dbcee8ff1dbc None None] MFA Rules not processed for user `b4b26ba3a9e04c38b52db2a46198a53c`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:34:54.561869 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-1f7b4f17-d342-4fe7-ae8c-dbcee8ff1dbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:34:54.672192 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 696/1399] 10.222.0.57 () {58 vars in 1040 bytes} [Wed Aug 10 08:34:54 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:34:54.682764 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b926e961-1665-46e6-b8fe-04459199265d None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:34:54.684347 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-b926e961-1665-46e6-b8fe-04459199265d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:34:54.729112 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:34:54.818350 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b926e961-1665-46e6-b8fe-04459199265d None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:34:54.922345 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9400faa7-7868-4d0f-b642-888f2b552345 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4b26ba3a9e04c38b52db2a46198a53c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:34:54.922958 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9400faa7-7868-4d0f-b642-888f2b552345 service swift] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:34:54.923241 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9400faa7-7868-4d0f-b642-888f2b552345 service swift] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:34:54.923517 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9400faa7-7868-4d0f-b642-888f2b552345 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:34:54.925289 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-9400faa7-7868-4d0f-b642-888f2b552345 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:34:54.955332 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:34:54.963752 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f2737a-efaa-4e1b-927a-d1186f10d306 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:34:54.964117 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f2737a-efaa-4e1b-927a-d1186f10d306 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:34:54.964305 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f2737a-efaa-4e1b-927a-d1186f10d306 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:34:54.965679 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 697/1400] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:34:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:34:54.970680 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29f3f96-f94f-4184-b720-832a527d0d97 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:34:54.970956 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29f3f96-f94f-4184-b720-832a527d0d97 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:34:54.971228 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29f3f96-f94f-4184-b720-832a527d0d97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:34:55.011388 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9400faa7-7868-4d0f-b642-888f2b552345 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:34:55.013039 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9400faa7-7868-4d0f-b642-888f2b552345 service swift] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:34:55.021983 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 704/1401] 10.222.0.57 () {60 vars in 1384 bytes} [Wed Aug 10 08:34:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 08:34:55.039787 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-a29f3f96-f94f-4184-b720-832a527d0d97 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:34:55.041136 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-a29f3f96-f94f-4184-b720-832a527d0d97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:34:55.097078 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:34:55.139003 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 698/1402] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:34:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 08:34:55.183705 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9400faa7-7868-4d0f-b642-888f2b552345 service swift] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:34:55.184906 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-9400faa7-7868-4d0f-b642-888f2b552345 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:34:55.276209 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9400faa7-7868-4d0f-b642-888f2b552345 service swift] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:34:55.348101 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:34:55.392078 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddef8b91-f799-4683-aace-7d7ccf4e2d00 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a943008193914f76a663d489bd73e873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:34:55.392676 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddef8b91-f799-4683-aace-7d7ccf4e2d00 service nova] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:34:55.392984 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddef8b91-f799-4683-aace-7d7ccf4e2d00 service nova] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:34:55.393251 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddef8b91-f799-4683-aace-7d7ccf4e2d00 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:34:55.394836 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-ddef8b91-f799-4683-aace-7d7ccf4e2d00 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:34:55.503922 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddef8b91-f799-4683-aace-7d7ccf4e2d00 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:34:55.505691 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddef8b91-f799-4683-aace-7d7ccf4e2d00 service nova] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:34:55.524567 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 705/1403] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:34:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:34:57.766839 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7fd35a-3aeb-4a8c-a5db-1661cb0c9a21 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:34:57.768036 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7fd35a-3aeb-4a8c-a5db-1661cb0c9a21 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:34:57.768362 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7fd35a-3aeb-4a8c-a5db-1661cb0c9a21 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:34:57.770099 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 699/1404] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:34:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:34:57.775673 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be499338-43e4-449e-b944-d445c783aeaf None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:34:57.776145 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be499338-43e4-449e-b944-d445c783aeaf None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:34:57.776250 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be499338-43e4-449e-b944-d445c783aeaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:34:57.848083 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-be499338-43e4-449e-b944-d445c783aeaf None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:34:57.849356 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-be499338-43e4-449e-b944-d445c783aeaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:34:57.886936 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:34:57.957388 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 706/1405] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:34:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:34:58.003590 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7fd35a-3aeb-4a8c-a5db-1661cb0c9a21 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:34:58.005186 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-9d7fd35a-3aeb-4a8c-a5db-1661cb0c9a21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:34:58.177642 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7fd35a-3aeb-4a8c-a5db-1661cb0c9a21 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:34:58.224833 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:34:58.312776 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb214141-112b-4f95-992f-cc83215495eb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a943008193914f76a663d489bd73e873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:34:58.313424 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb214141-112b-4f95-992f-cc83215495eb service nova] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:34:58.313706 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb214141-112b-4f95-992f-cc83215495eb service nova] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:34:58.313983 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb214141-112b-4f95-992f-cc83215495eb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:34:58.315668 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-eb214141-112b-4f95-992f-cc83215495eb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:34:58.426972 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb214141-112b-4f95-992f-cc83215495eb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:34:58.428635 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb214141-112b-4f95-992f-cc83215495eb service nova] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:34:58.450692 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 700/1406] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:34:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:35:00.642132 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d16f91d-b65d-4472-9302-edb69b8eafec None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:35:00.642697 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d16f91d-b65d-4472-9302-edb69b8eafec None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:35:00.642697 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d16f91d-b65d-4472-9302-edb69b8eafec None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:35:00.644168 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 707/1407] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:35:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:35:00.649591 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4f889e-9558-4989-92c9-546eb0121e4b None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:35:00.650091 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4f889e-9558-4989-92c9-546eb0121e4b None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:35:00.650185 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4f889e-9558-4989-92c9-546eb0121e4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:35:00.714246 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:35:00.723254 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-1f4f889e-9558-4989-92c9-546eb0121e4b None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:35:00.737295 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-1f4f889e-9558-4989-92c9-546eb0121e4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:35:00.835712 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 701/1408] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:35:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:35:00.881224 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d16f91d-b65d-4472-9302-edb69b8eafec None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:35:00.882985 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-0d16f91d-b65d-4472-9302-edb69b8eafec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:35:01.058549 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d16f91d-b65d-4472-9302-edb69b8eafec None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:35:01.077508 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:35:01.188198 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1322aa-5fba-48ff-9b25-a708c90e2310 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a943008193914f76a663d489bd73e873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:35:01.188975 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1322aa-5fba-48ff-9b25-a708c90e2310 service nova] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:35:01.189259 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1322aa-5fba-48ff-9b25-a708c90e2310 service nova] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:35:01.189532 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1322aa-5fba-48ff-9b25-a708c90e2310 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:35:01.191136 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-4d1322aa-5fba-48ff-9b25-a708c90e2310 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:35:01.299946 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1322aa-5fba-48ff-9b25-a708c90e2310 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:35:01.301598 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1322aa-5fba-48ff-9b25-a708c90e2310 service nova] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:35:01.322117 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 708/1409] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:35:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:35:10.833485 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=44 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:35:11.318353 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=90 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:35:18.652192 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec913d19-26c6-4a0a-a667-d8df9f4a41b6 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:35:18.652762 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec913d19-26c6-4a0a-a667-d8df9f4a41b6 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:35:18.652762 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec913d19-26c6-4a0a-a667-d8df9f4a41b6 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:35:18.654319 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 702/1410] 10.222.0.57 () {54 vars in 918 bytes} [Wed Aug 10 08:35:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:35:18.659479 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a171c81-7200-40a9-abf2-a7604eb85887 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:35:18.659960 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a171c81-7200-40a9-abf2-a7604eb85887 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:35:18.666252 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a171c81-7200-40a9-abf2-a7604eb85887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:35:18.734915 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-2a171c81-7200-40a9-abf2-a7604eb85887 None None] MFA Rules not processed for user `2912307c56594125b157f1cfdf7c96a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:35:18.736123 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-2a171c81-7200-40a9-abf2-a7604eb85887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:35:18.842743 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 709/1411] 10.222.0.57 () {58 vars in 1015 bytes} [Wed Aug 10 08:35:18 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 08:35:18.857531 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f15097-a02c-4269-a97a-1ce3e1e575a8 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:35:18.857821 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f15097-a02c-4269-a97a-1ce3e1e575a8 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:35:18.858101 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f15097-a02c-4269-a97a-1ce3e1e575a8 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:35:18.859449 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 703/1412] 10.222.0.57 () {54 vars in 918 bytes} [Wed Aug 10 08:35:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:35:18.863824 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72be7d7-aa2a-4460-87ea-724b0e580f85 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:35:18.864108 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72be7d7-aa2a-4460-87ea-724b0e580f85 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:35:18.864379 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72be7d7-aa2a-4460-87ea-724b0e580f85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:35:18.936034 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:35:18.939455 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-f72be7d7-aa2a-4460-87ea-724b0e580f85 None None] MFA Rules not processed for user `a943008193914f76a663d489bd73e873`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:35:18.946525 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-f72be7d7-aa2a-4460-87ea-724b0e580f85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:35:19.057895 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 710/1413] 10.222.0.57 () {58 vars in 1015 bytes} [Wed Aug 10 08:35:18 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:35:21.538915 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63835bba-34df-4c91-8196-74d8ce8c0f8a None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:35:21.539477 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63835bba-34df-4c91-8196-74d8ce8c0f8a None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:35:21.539477 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63835bba-34df-4c91-8196-74d8ce8c0f8a None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:35:21.540775 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 704/1414] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 08:35:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:35:21.545901 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5159e7-40a5-408b-9e7b-be5d7c0412b3 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:35:21.546303 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5159e7-40a5-408b-9e7b-be5d7c0412b3 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:35:21.546429 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5159e7-40a5-408b-9e7b-be5d7c0412b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:35:21.612748 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-6a5159e7-40a5-408b-9e7b-be5d7c0412b3 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:35:21.613998 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-6a5159e7-40a5-408b-9e7b-be5d7c0412b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:35:21.668188 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:35:21.709104 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 711/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 08:35:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:35:21.751482 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63835bba-34df-4c91-8196-74d8ce8c0f8a None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:35:21.753295 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-63835bba-34df-4c91-8196-74d8ce8c0f8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:35:21.897886 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63835bba-34df-4c91-8196-74d8ce8c0f8a None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:35:21.968607 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:35:22.008411 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5790600-7ab4-4d64-ba1d-6dc10f0a51bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a943008193914f76a663d489bd73e873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:35:22.009055 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5790600-7ab4-4d64-ba1d-6dc10f0a51bc service nova] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:35:22.009556 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5790600-7ab4-4d64-ba1d-6dc10f0a51bc service nova] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:35:22.009876 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5790600-7ab4-4d64-ba1d-6dc10f0a51bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:35:22.012025 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-a5790600-7ab4-4d64-ba1d-6dc10f0a51bc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:35:22.101278 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5790600-7ab4-4d64-ba1d-6dc10f0a51bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:35:22.102868 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5790600-7ab4-4d64-ba1d-6dc10f0a51bc service nova] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:35:22.120442 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 705/1416] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:35:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:35:23.023900 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b7f54f-3f61-4469-846d-29b8aa2f69bc None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:35:23.024476 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b7f54f-3f61-4469-846d-29b8aa2f69bc None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:35:23.024476 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b7f54f-3f61-4469-846d-29b8aa2f69bc None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:35:23.025823 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 712/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 08:35:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:35:23.030797 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135c59fa-dacc-446a-ae3b-21357d950e58 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:35:23.031245 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135c59fa-dacc-446a-ae3b-21357d950e58 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:35:23.031355 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135c59fa-dacc-446a-ae3b-21357d950e58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:35:23.106934 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-135c59fa-dacc-446a-ae3b-21357d950e58 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:35:23.108140 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-135c59fa-dacc-446a-ae3b-21357d950e58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:35:23.135230 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:35:23.204068 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 706/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 08:35:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:35:23.355392 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b7f54f-3f61-4469-846d-29b8aa2f69bc None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:35:23.356841 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-40b7f54f-3f61-4469-846d-29b8aa2f69bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:35:23.501445 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b7f54f-3f61-4469-846d-29b8aa2f69bc None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:35:23.542104 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:35:23.614129 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dc6ed3-bdd6-4864-b8cc-c131c3d6354f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a943008193914f76a663d489bd73e873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:35:23.614650 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dc6ed3-bdd6-4864-b8cc-c131c3d6354f service nova] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:35:23.614925 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dc6ed3-bdd6-4864-b8cc-c131c3d6354f service nova] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:35:23.615188 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dc6ed3-bdd6-4864-b8cc-c131c3d6354f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:35:23.617760 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-a2dc6ed3-bdd6-4864-b8cc-c131c3d6354f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:35:23.706277 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dc6ed3-bdd6-4864-b8cc-c131c3d6354f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:35:23.707869 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dc6ed3-bdd6-4864-b8cc-c131c3d6354f service nova] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:35:23.726008 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 713/1419] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:35:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:35:24.607048 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7ddc3e-5da4-426a-b934-f333fe095f88 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:35:24.607582 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7ddc3e-5da4-426a-b934-f333fe095f88 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:35:24.607582 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7ddc3e-5da4-426a-b934-f333fe095f88 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:35:24.608969 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 707/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 08:35:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:35:24.613874 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce28358f-b686-4147-a8f7-47370b5a5034 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:35:24.614324 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce28358f-b686-4147-a8f7-47370b5a5034 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:35:24.614423 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce28358f-b686-4147-a8f7-47370b5a5034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:35:24.669748 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:35:24.692815 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-ce28358f-b686-4147-a8f7-47370b5a5034 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:35:24.694633 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-ce28358f-b686-4147-a8f7-47370b5a5034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:35:24.790498 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 714/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 08:35:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:35:24.940988 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7ddc3e-5da4-426a-b934-f333fe095f88 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:35:24.942432 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-ad7ddc3e-5da4-426a-b934-f333fe095f88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:35:25.089117 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7ddc3e-5da4-426a-b934-f333fe095f88 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:35:25.101033 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:35:25.199734 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797d6002-514d-494d-b2df-de993cb71ee2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a943008193914f76a663d489bd73e873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:35:25.200254 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797d6002-514d-494d-b2df-de993cb71ee2 service nova] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:35:25.200528 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797d6002-514d-494d-b2df-de993cb71ee2 service nova] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:35:25.200793 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797d6002-514d-494d-b2df-de993cb71ee2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:35:25.202470 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-797d6002-514d-494d-b2df-de993cb71ee2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:35:25.291117 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797d6002-514d-494d-b2df-de993cb71ee2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:35:25.292709 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797d6002-514d-494d-b2df-de993cb71ee2 service nova] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:35:25.311067 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 708/1422] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:35:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:35:26.171467 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a2d8a9-147b-49f2-babb-ccaaec3089ce None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:35:26.172003 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a2d8a9-147b-49f2-babb-ccaaec3089ce None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:35:26.172003 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a2d8a9-147b-49f2-babb-ccaaec3089ce None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:35:26.173443 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 715/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 08:35:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:35:26.179454 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4269acec-e921-461f-a6a3-9f965702c6de None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:35:26.179888 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4269acec-e921-461f-a6a3-9f965702c6de None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:35:26.180041 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4269acec-e921-461f-a6a3-9f965702c6de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:35:26.199674 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:35:26.258133 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-4269acec-e921-461f-a6a3-9f965702c6de None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:35:26.259318 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-4269acec-e921-461f-a6a3-9f965702c6de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:35:26.354344 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 709/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 08:35:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 08:35:26.505183 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a2d8a9-147b-49f2-babb-ccaaec3089ce None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:35:26.506630 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-45a2d8a9-147b-49f2-babb-ccaaec3089ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:35:26.614428 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:35:26.650474 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a2d8a9-147b-49f2-babb-ccaaec3089ce None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:35:26.761934 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cb04b4-7129-4f5e-825b-32a91ab36e4c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a943008193914f76a663d489bd73e873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:35:26.762449 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cb04b4-7129-4f5e-825b-32a91ab36e4c service nova] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:35:26.762723 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cb04b4-7129-4f5e-825b-32a91ab36e4c service nova] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:35:26.762989 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cb04b4-7129-4f5e-825b-32a91ab36e4c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:35:26.764563 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-f3cb04b4-7129-4f5e-825b-32a91ab36e4c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:35:26.839503 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:35:26.853266 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cb04b4-7129-4f5e-825b-32a91ab36e4c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:35:26.854861 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cb04b4-7129-4f5e-825b-32a91ab36e4c service nova] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:35:26.873351 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 716/1425] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:35:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:35:27.738975 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1841d3d7-a1e2-43e3-960b-43de4fe6380d None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:35:27.739540 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1841d3d7-a1e2-43e3-960b-43de4fe6380d None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:35:27.739540 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1841d3d7-a1e2-43e3-960b-43de4fe6380d None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:35:27.741525 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 710/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 08:35:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:35:27.746382 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6d67e3-0e4a-47f2-8614-84f539752e1f None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:35:27.746790 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6d67e3-0e4a-47f2-8614-84f539752e1f None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:35:27.746920 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6d67e3-0e4a-47f2-8614-84f539752e1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:35:27.824027 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-0b6d67e3-0e4a-47f2-8614-84f539752e1f None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:35:27.825294 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-0b6d67e3-0e4a-47f2-8614-84f539752e1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:35:28.049085 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 717/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 08:35:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:35:28.199579 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1841d3d7-a1e2-43e3-960b-43de4fe6380d None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:35:28.201184 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-1841d3d7-a1e2-43e3-960b-43de4fe6380d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:35:28.285519 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:35:28.358548 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1841d3d7-a1e2-43e3-960b-43de4fe6380d None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:35:28.474291 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526e1dc1-be3d-4f5b-ac3b-d096760f12c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a943008193914f76a663d489bd73e873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:35:28.474983 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526e1dc1-be3d-4f5b-ac3b-d096760f12c8 service nova] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:35:28.475287 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526e1dc1-be3d-4f5b-ac3b-d096760f12c8 service nova] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:35:28.475643 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526e1dc1-be3d-4f5b-ac3b-d096760f12c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:35:28.477268 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-526e1dc1-be3d-4f5b-ac3b-d096760f12c8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:35:28.525315 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:35:28.565766 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526e1dc1-be3d-4f5b-ac3b-d096760f12c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:35:28.567351 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526e1dc1-be3d-4f5b-ac3b-d096760f12c8 service nova] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:35:28.586462 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 711/1428] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:35:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:35:29.461326 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a28bcab-03ab-4f55-83a8-507907956840 None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:35:29.461898 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a28bcab-03ab-4f55-83a8-507907956840 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:35:29.461898 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a28bcab-03ab-4f55-83a8-507907956840 None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:35:29.463189 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 718/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 08:35:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:35:29.468290 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58821aac-3d4a-4f79-a583-3a54bf84213c None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:35:29.468716 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58821aac-3d4a-4f79-a583-3a54bf84213c None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:35:29.468820 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58821aac-3d4a-4f79-a583-3a54bf84213c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:35:29.544134 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-58821aac-3d4a-4f79-a583-3a54bf84213c None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:35:29.545398 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-58821aac-3d4a-4f79-a583-3a54bf84213c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:35:29.641226 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 712/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 08:35:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:35:29.795829 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a28bcab-03ab-4f55-83a8-507907956840 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:35:29.797478 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-1a28bcab-03ab-4f55-83a8-507907956840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:35:29.845630 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:35:29.941081 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a28bcab-03ab-4f55-83a8-507907956840 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:35:30.058880 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7f4541-fd10-43b8-9881-b32863db1059 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a943008193914f76a663d489bd73e873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:35:30.059416 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7f4541-fd10-43b8-9881-b32863db1059 service nova] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:35:30.059695 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7f4541-fd10-43b8-9881-b32863db1059 service nova] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:35:30.059969 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7f4541-fd10-43b8-9881-b32863db1059 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:35:30.061661 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-7c7f4541-fd10-43b8-9881-b32863db1059 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:35:30.068815 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:35:30.150748 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7f4541-fd10-43b8-9881-b32863db1059 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:35:30.152338 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7f4541-fd10-43b8-9881-b32863db1059 service nova] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:35:30.170000 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 719/1431] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:35:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:35:31.016761 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a73d69e-13a6-44ff-91e0-c3f988729263 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:35:31.017295 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a73d69e-13a6-44ff-91e0-c3f988729263 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:35:31.017295 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a73d69e-13a6-44ff-91e0-c3f988729263 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:35:31.018667 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 713/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 08:35:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:35:31.024234 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2528c0c9-858d-4841-abe6-87b7f78c5564 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:35:31.024669 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2528c0c9-858d-4841-abe6-87b7f78c5564 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:35:31.024775 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2528c0c9-858d-4841-abe6-87b7f78c5564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:35:31.099133 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-2528c0c9-858d-4841-abe6-87b7f78c5564 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:35:31.100309 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-2528c0c9-858d-4841-abe6-87b7f78c5564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:35:31.167041 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:35:31.195086 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 720/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 08:35:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:35:31.348595 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a73d69e-13a6-44ff-91e0-c3f988729263 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:35:31.350171 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-3a73d69e-13a6-44ff-91e0-c3f988729263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:35:31.368105 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:35:31.497624 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a73d69e-13a6-44ff-91e0-c3f988729263 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:35:31.584759 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:35:31.611481 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c43a06-b1e3-4542-b065-227167829662 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a943008193914f76a663d489bd73e873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:35:31.612081 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c43a06-b1e3-4542-b065-227167829662 service nova] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:35:31.612356 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c43a06-b1e3-4542-b065-227167829662 service nova] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:35:31.612621 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c43a06-b1e3-4542-b065-227167829662 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:35:31.614266 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-74c43a06-b1e3-4542-b065-227167829662 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:35:31.705940 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c43a06-b1e3-4542-b065-227167829662 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:35:31.707536 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c43a06-b1e3-4542-b065-227167829662 service nova] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:35:31.725390 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 714/1434] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:35:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:35:32.586599 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23b0184-121c-48ae-a0b9-f1e2287722ed None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:35:32.587159 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23b0184-121c-48ae-a0b9-f1e2287722ed None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:35:32.587159 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23b0184-121c-48ae-a0b9-f1e2287722ed None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:35:32.588493 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 721/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 08:35:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:35:32.593541 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc29e6dd-e7c3-4fd5-b525-2168091cba08 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:35:32.593930 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc29e6dd-e7c3-4fd5-b525-2168091cba08 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:35:32.594079 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc29e6dd-e7c3-4fd5-b525-2168091cba08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:35:32.669771 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-bc29e6dd-e7c3-4fd5-b525-2168091cba08 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:35:32.670991 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-bc29e6dd-e7c3-4fd5-b525-2168091cba08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:35:32.713114 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:35:32.768757 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 715/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 08:35:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:35:32.920440 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23b0184-121c-48ae-a0b9-f1e2287722ed None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:35:32.921909 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-a23b0184-121c-48ae-a0b9-f1e2287722ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:35:33.065953 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23b0184-121c-48ae-a0b9-f1e2287722ed None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:35:33.119013 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:35:33.178359 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af74a154-29e8-4e17-8453-6a890e2cdedb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a943008193914f76a663d489bd73e873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:35:33.178880 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af74a154-29e8-4e17-8453-6a890e2cdedb service nova] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:35:33.179774 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af74a154-29e8-4e17-8453-6a890e2cdedb service nova] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:35:33.180449 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af74a154-29e8-4e17-8453-6a890e2cdedb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:35:33.182067 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-af74a154-29e8-4e17-8453-6a890e2cdedb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:35:33.270537 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af74a154-29e8-4e17-8453-6a890e2cdedb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:35:33.272110 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af74a154-29e8-4e17-8453-6a890e2cdedb service nova] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:35:33.290311 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 722/1437] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:35:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:35:34.172015 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cb035c-fbc2-46da-b953-1faa51535770 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:35:34.177236 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cb035c-fbc2-46da-b953-1faa51535770 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:35:34.177236 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cb035c-fbc2-46da-b953-1faa51535770 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:35:34.177236 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 716/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 08:35:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:35:34.178960 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f042eb73-ec67-4540-9d5a-c638a84ada54 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:35:34.179605 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f042eb73-ec67-4540-9d5a-c638a84ada54 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:35:34.179879 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f042eb73-ec67-4540-9d5a-c638a84ada54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:35:34.262302 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-f042eb73-ec67-4540-9d5a-c638a84ada54 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:35:34.263512 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-f042eb73-ec67-4540-9d5a-c638a84ada54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:35:34.265564 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:35:34.362063 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 723/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 08:35:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:35:34.516777 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cb035c-fbc2-46da-b953-1faa51535770 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:35:34.518265 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-d9cb035c-fbc2-46da-b953-1faa51535770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:35:34.663759 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cb035c-fbc2-46da-b953-1faa51535770 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:35:34.690256 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:35:34.775953 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4435b45c-50f9-42ba-83ee-3ffc90699a7e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a943008193914f76a663d489bd73e873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:35:34.776482 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4435b45c-50f9-42ba-83ee-3ffc90699a7e service nova] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:35:34.776763 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4435b45c-50f9-42ba-83ee-3ffc90699a7e service nova] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:35:34.777075 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4435b45c-50f9-42ba-83ee-3ffc90699a7e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:35:34.778694 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-4435b45c-50f9-42ba-83ee-3ffc90699a7e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:35:34.867287 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4435b45c-50f9-42ba-83ee-3ffc90699a7e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:35:34.868892 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4435b45c-50f9-42ba-83ee-3ffc90699a7e service nova] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:35:34.886577 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 717/1440] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:35:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:35:36.763468 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8682425-4759-421c-98ff-3fef76bb6bc4 None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:35:36.764067 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8682425-4759-421c-98ff-3fef76bb6bc4 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:35:36.764067 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8682425-4759-421c-98ff-3fef76bb6bc4 None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:35:36.765441 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 724/1441] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 08:35:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:35:36.770823 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1218b5e5-7300-4040-b7cf-0f8713fa5898 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:35:36.771211 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1218b5e5-7300-4040-b7cf-0f8713fa5898 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:35:36.771373 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1218b5e5-7300-4040-b7cf-0f8713fa5898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:35:36.812257 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:35:36.836301 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-1218b5e5-7300-4040-b7cf-0f8713fa5898 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:35:36.837538 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-1218b5e5-7300-4040-b7cf-0f8713fa5898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:35:36.936863 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 718/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 08:35:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:35:36.976511 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8682425-4759-421c-98ff-3fef76bb6bc4 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:35:36.978114 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-d8682425-4759-421c-98ff-3fef76bb6bc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:35:37.081791 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8682425-4759-421c-98ff-3fef76bb6bc4 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:35:37.114377 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:35:37.156170 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30841b32-e68b-4fbb-b2a3-c89192a66435 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:35:37.156768 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30841b32-e68b-4fbb-b2a3-c89192a66435 service neutron] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:35:37.157078 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30841b32-e68b-4fbb-b2a3-c89192a66435 service neutron] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:35:37.157351 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30841b32-e68b-4fbb-b2a3-c89192a66435 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:35:37.159046 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-30841b32-e68b-4fbb-b2a3-c89192a66435 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:35:37.267894 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30841b32-e68b-4fbb-b2a3-c89192a66435 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:35:37.269371 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30841b32-e68b-4fbb-b2a3-c89192a66435 service neutron] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:35:37.288667 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 725/1443] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:35:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:35:39.518531 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b34ce2-5529-4f41-a731-9b8b3b978d82 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:35:39.519093 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b34ce2-5529-4f41-a731-9b8b3b978d82 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:35:39.519093 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b34ce2-5529-4f41-a731-9b8b3b978d82 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:35:39.520560 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 719/1444] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 08:35:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:35:39.526718 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbebed9-848c-40ea-ac98-dbc2742bfaa4 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:35:39.527142 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbebed9-848c-40ea-ac98-dbc2742bfaa4 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:35:39.527256 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbebed9-848c-40ea-ac98-dbc2742bfaa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:35:39.593531 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-abbebed9-848c-40ea-ac98-dbc2742bfaa4 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:35:39.594714 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-abbebed9-848c-40ea-ac98-dbc2742bfaa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:35:39.607542 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:35:39.690682 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 726/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 08:35:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:35:39.729222 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b34ce2-5529-4f41-a731-9b8b3b978d82 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:35:39.730766 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-95b34ce2-5529-4f41-a731-9b8b3b978d82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:35:39.833680 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b34ce2-5529-4f41-a731-9b8b3b978d82 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:35:39.846174 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:35:39.908874 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69d93cb-6b58-4191-b1cb-e21b6ce2a606 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:35:39.909510 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69d93cb-6b58-4191-b1cb-e21b6ce2a606 service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:35:39.909798 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69d93cb-6b58-4191-b1cb-e21b6ce2a606 service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:35:39.910089 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69d93cb-6b58-4191-b1cb-e21b6ce2a606 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:35:39.911669 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-e69d93cb-6b58-4191-b1cb-e21b6ce2a606 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:35:40.024813 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69d93cb-6b58-4191-b1cb-e21b6ce2a606 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:35:40.026294 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69d93cb-6b58-4191-b1cb-e21b6ce2a606 service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:35:40.044975 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 720/1446] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:35:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:35:42.411300 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b315ba80-cfe9-444a-8348-d35ade884464 None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:35:42.411716 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b315ba80-cfe9-444a-8348-d35ade884464 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:35:42.411849 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b315ba80-cfe9-444a-8348-d35ade884464 None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:35:42.413884 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 727/1447] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 08:35:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:35:42.419630 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b514d2e-b522-4578-a17f-3a4e01e1ba66 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:35:42.420044 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b514d2e-b522-4578-a17f-3a4e01e1ba66 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:35:42.420169 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b514d2e-b522-4578-a17f-3a4e01e1ba66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:35:42.469208 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:35:42.487345 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-9b514d2e-b522-4578-a17f-3a4e01e1ba66 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:35:42.488756 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-9b514d2e-b522-4578-a17f-3a4e01e1ba66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:35:42.707511 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 721/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 08:35:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:35:42.747265 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b315ba80-cfe9-444a-8348-d35ade884464 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:35:42.748826 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-b315ba80-cfe9-444a-8348-d35ade884464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:35:42.855148 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b315ba80-cfe9-444a-8348-d35ade884464 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:35:42.895803 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:35:42.930051 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c1f49b-5b19-476f-8d85-c87b0ed3a17e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:35:42.930654 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c1f49b-5b19-476f-8d85-c87b0ed3a17e service neutron] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:35:42.930933 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c1f49b-5b19-476f-8d85-c87b0ed3a17e service neutron] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:35:42.931204 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c1f49b-5b19-476f-8d85-c87b0ed3a17e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:35:42.932801 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-e0c1f49b-5b19-476f-8d85-c87b0ed3a17e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:35:43.042091 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c1f49b-5b19-476f-8d85-c87b0ed3a17e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:35:43.043460 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c1f49b-5b19-476f-8d85-c87b0ed3a17e service neutron] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:35:43.062256 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 728/1449] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:35:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:35:45.211749 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2327b04d-7d8d-475a-a2bc-db3e638ca32c None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:35:45.212328 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2327b04d-7d8d-475a-a2bc-db3e638ca32c None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:35:45.212328 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2327b04d-7d8d-475a-a2bc-db3e638ca32c None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:35:45.213656 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 722/1450] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 08:35:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:35:45.218890 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9840d590-3c9f-4f29-94b1-c203e7f6f751 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:35:45.219324 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9840d590-3c9f-4f29-94b1-c203e7f6f751 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:35:45.219468 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9840d590-3c9f-4f29-94b1-c203e7f6f751 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:35:45.284337 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-9840d590-3c9f-4f29-94b1-c203e7f6f751 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:35:45.285578 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-9840d590-3c9f-4f29-94b1-c203e7f6f751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:35:45.306691 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:35:45.383076 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 729/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 08:35:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:35:45.422566 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2327b04d-7d8d-475a-a2bc-db3e638ca32c None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:35:45.424086 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-2327b04d-7d8d-475a-a2bc-db3e638ca32c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:35:45.526117 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2327b04d-7d8d-475a-a2bc-db3e638ca32c None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:35:45.546092 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:35:45.598019 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9bc9e5-abcd-4439-b687-fadb5eddec66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:35:45.598607 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9bc9e5-abcd-4439-b687-fadb5eddec66 service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:35:45.598879 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9bc9e5-abcd-4439-b687-fadb5eddec66 service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:35:45.599141 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9bc9e5-abcd-4439-b687-fadb5eddec66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:35:45.600723 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-ea9bc9e5-abcd-4439-b687-fadb5eddec66 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:35:45.706578 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9bc9e5-abcd-4439-b687-fadb5eddec66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:35:45.707985 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9bc9e5-abcd-4439-b687-fadb5eddec66 service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:35:45.726209 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 723/1452] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:35:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:35:47.799067 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbd515d-9d9e-48fa-8720-c2360052aeeb None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:35:47.799636 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbd515d-9d9e-48fa-8720-c2360052aeeb None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:35:47.799636 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbd515d-9d9e-48fa-8720-c2360052aeeb None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:35:47.800968 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 730/1453] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 08:35:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:35:47.806462 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db370107-fc7d-4545-bf6e-bb4477c7a8de None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:35:47.806875 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db370107-fc7d-4545-bf6e-bb4477c7a8de None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:35:47.807023 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db370107-fc7d-4545-bf6e-bb4477c7a8de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:35:47.871992 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:35:47.874919 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-db370107-fc7d-4545-bf6e-bb4477c7a8de None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:35:47.876027 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-db370107-fc7d-4545-bf6e-bb4477c7a8de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:35:47.974409 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 724/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 08:35:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 08:35:48.014234 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbd515d-9d9e-48fa-8720-c2360052aeeb None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:35:48.015761 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-0bbd515d-9d9e-48fa-8720-c2360052aeeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:35:48.118206 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbd515d-9d9e-48fa-8720-c2360052aeeb None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:35:48.166105 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:35:48.191769 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76870d13-7be0-484c-be0e-8c26d6abf49c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:35:48.192373 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76870d13-7be0-484c-be0e-8c26d6abf49c service neutron] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:35:48.192651 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76870d13-7be0-484c-be0e-8c26d6abf49c service neutron] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:35:48.192940 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76870d13-7be0-484c-be0e-8c26d6abf49c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:35:48.194572 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-76870d13-7be0-484c-be0e-8c26d6abf49c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:35:48.301570 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76870d13-7be0-484c-be0e-8c26d6abf49c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:35:48.303003 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76870d13-7be0-484c-be0e-8c26d6abf49c service neutron] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:35:48.320802 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 731/1455] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:35:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:35:50.401257 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d237e903-2618-414c-ae55-c447db07f973 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:35:50.401828 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d237e903-2618-414c-ae55-c447db07f973 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:35:50.401828 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d237e903-2618-414c-ae55-c447db07f973 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:35:50.403461 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 725/1456] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 08:35:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:35:50.408508 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8a914e-c18b-4003-a2d1-202755a762bb None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:35:50.408950 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8a914e-c18b-4003-a2d1-202755a762bb None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:35:50.409095 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8a914e-c18b-4003-a2d1-202755a762bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:35:50.473771 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-3e8a914e-c18b-4003-a2d1-202755a762bb None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:35:50.474973 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-3e8a914e-c18b-4003-a2d1-202755a762bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:35:50.503906 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:35:50.572765 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 732/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 08:35:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:35:50.610739 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d237e903-2618-414c-ae55-c447db07f973 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:35:50.612350 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-d237e903-2618-414c-ae55-c447db07f973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:35:50.713742 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d237e903-2618-414c-ae55-c447db07f973 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:35:50.741799 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:35:50.786369 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d18424-d510-4db3-93e3-22cde883d28a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:35:50.786961 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d18424-d510-4db3-93e3-22cde883d28a service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:35:50.787234 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d18424-d510-4db3-93e3-22cde883d28a service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:35:50.787496 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d18424-d510-4db3-93e3-22cde883d28a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:35:50.790117 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-97d18424-d510-4db3-93e3-22cde883d28a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:35:50.897834 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d18424-d510-4db3-93e3-22cde883d28a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:35:50.899289 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d18424-d510-4db3-93e3-22cde883d28a service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:35:50.917092 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 726/1458] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:35:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:35:52.996229 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42f3b5d-8912-4ae7-8937-9861521e81e2 None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:35:52.996803 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42f3b5d-8912-4ae7-8937-9861521e81e2 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:35:52.996803 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42f3b5d-8912-4ae7-8937-9861521e81e2 None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:35:52.998144 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 733/1459] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 08:35:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:35:53.003714 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725daef0-2f33-497d-950b-aa1947e01410 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:35:53.004142 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725daef0-2f33-497d-950b-aa1947e01410 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:35:53.004248 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725daef0-2f33-497d-950b-aa1947e01410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:35:53.071456 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-725daef0-2f33-497d-950b-aa1947e01410 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:35:53.072694 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-725daef0-2f33-497d-950b-aa1947e01410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:35:53.080958 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:35:53.169205 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 727/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 08:35:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:35:53.207318 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42f3b5d-8912-4ae7-8937-9861521e81e2 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:35:53.208848 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-e42f3b5d-8912-4ae7-8937-9861521e81e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:35:53.310597 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42f3b5d-8912-4ae7-8937-9861521e81e2 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:35:53.364073 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:35:53.382529 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32faf35-8c66-4413-82d0-b47a8a1faef5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:35:53.383122 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32faf35-8c66-4413-82d0-b47a8a1faef5 service neutron] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:35:53.383404 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32faf35-8c66-4413-82d0-b47a8a1faef5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:35:53.383672 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32faf35-8c66-4413-82d0-b47a8a1faef5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:35:53.385302 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-e32faf35-8c66-4413-82d0-b47a8a1faef5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:35:53.492970 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32faf35-8c66-4413-82d0-b47a8a1faef5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:35:53.494406 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32faf35-8c66-4413-82d0-b47a8a1faef5 service neutron] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:35:53.513914 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 734/1461] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:35:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:35:55.558915 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9214dc6f-3e0d-49cf-b5e4-e58d1f203124 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:35:55.559473 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9214dc6f-3e0d-49cf-b5e4-e58d1f203124 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:35:55.559473 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9214dc6f-3e0d-49cf-b5e4-e58d1f203124 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:35:55.560790 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 728/1462] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 08:35:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:35:55.566159 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9317712-341f-41e4-ae23-5f66f5844a13 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:35:55.566593 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9317712-341f-41e4-ae23-5f66f5844a13 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:35:55.566693 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9317712-341f-41e4-ae23-5f66f5844a13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:35:55.632754 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-b9317712-341f-41e4-ae23-5f66f5844a13 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:35:55.634215 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-b9317712-341f-41e4-ae23-5f66f5844a13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:35:55.672476 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:35:55.732507 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 735/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 08:35:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:35:55.769877 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9214dc6f-3e0d-49cf-b5e4-e58d1f203124 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:35:55.771427 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-9214dc6f-3e0d-49cf-b5e4-e58d1f203124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:35:55.873910 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9214dc6f-3e0d-49cf-b5e4-e58d1f203124 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:35:55.910980 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:35:55.946648 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e029a6c-68b0-4589-ae6c-35e2bd9cead0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:35:55.947176 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e029a6c-68b0-4589-ae6c-35e2bd9cead0 service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:35:55.947462 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e029a6c-68b0-4589-ae6c-35e2bd9cead0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:35:55.947710 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e029a6c-68b0-4589-ae6c-35e2bd9cead0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:35:55.949307 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-8e029a6c-68b0-4589-ae6c-35e2bd9cead0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:35:56.054376 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e029a6c-68b0-4589-ae6c-35e2bd9cead0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:35:56.055791 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e029a6c-68b0-4589-ae6c-35e2bd9cead0 service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:35:56.074190 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 729/1464] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:35:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:35:58.092229 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632209f2-6131-4e37-8cf5-1a4db73cdc7c None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:35:58.092766 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632209f2-6131-4e37-8cf5-1a4db73cdc7c None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:35:58.092766 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632209f2-6131-4e37-8cf5-1a4db73cdc7c None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:35:58.094164 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 736/1465] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 08:35:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:35:58.099371 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033a898c-53cd-43ba-beb4-f4ddfb263b93 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:35:58.099808 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033a898c-53cd-43ba-beb4-f4ddfb263b93 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:35:58.099917 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033a898c-53cd-43ba-beb4-f4ddfb263b93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:35:58.166570 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-033a898c-53cd-43ba-beb4-f4ddfb263b93 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:35:58.167755 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-033a898c-53cd-43ba-beb4-f4ddfb263b93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:35:58.185707 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:35:58.265322 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 730/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 08:35:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:35:58.301965 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632209f2-6131-4e37-8cf5-1a4db73cdc7c None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:35:58.304013 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-632209f2-6131-4e37-8cf5-1a4db73cdc7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:35:58.408373 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632209f2-6131-4e37-8cf5-1a4db73cdc7c None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:35:58.475181 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:35:58.482008 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ffb872-85df-435f-bbc1-1a750b397b49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:35:58.482610 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ffb872-85df-435f-bbc1-1a750b397b49 service neutron] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:35:58.482889 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ffb872-85df-435f-bbc1-1a750b397b49 service neutron] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:35:58.483537 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ffb872-85df-435f-bbc1-1a750b397b49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:35:58.484767 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-91ffb872-85df-435f-bbc1-1a750b397b49 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:35:58.592940 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ffb872-85df-435f-bbc1-1a750b397b49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:35:58.594390 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ffb872-85df-435f-bbc1-1a750b397b49 service neutron] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:35:58.611943 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 737/1467] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:35:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:36:00.618535 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fc31d2-9a6a-46ba-b129-0e00988943c4 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:36:00.619146 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fc31d2-9a6a-46ba-b129-0e00988943c4 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:36:00.619146 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fc31d2-9a6a-46ba-b129-0e00988943c4 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:36:00.620414 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 731/1468] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 08:36:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:36:00.625672 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2317eb8f-371c-4d57-b65e-759e46e4d22d None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:36:00.626152 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2317eb8f-371c-4d57-b65e-759e46e4d22d None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:36:00.626261 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2317eb8f-371c-4d57-b65e-759e46e4d22d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:36:00.692339 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-2317eb8f-371c-4d57-b65e-759e46e4d22d None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:36:00.693561 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-2317eb8f-371c-4d57-b65e-759e46e4d22d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:36:00.740272 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:36:00.788315 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 738/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 08:36:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:36:00.824263 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fc31d2-9a6a-46ba-b129-0e00988943c4 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:36:00.825947 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-f7fc31d2-9a6a-46ba-b129-0e00988943c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:36:00.930307 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fc31d2-9a6a-46ba-b129-0e00988943c4 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:36:00.978041 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:36:01.003724 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d09d0c4-2f73-4254-8bcb-b04fe858333d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:36:01.004320 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d09d0c4-2f73-4254-8bcb-b04fe858333d service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:36:01.004591 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d09d0c4-2f73-4254-8bcb-b04fe858333d service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:36:01.004866 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d09d0c4-2f73-4254-8bcb-b04fe858333d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:36:01.006665 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-5d09d0c4-2f73-4254-8bcb-b04fe858333d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:36:01.113882 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d09d0c4-2f73-4254-8bcb-b04fe858333d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:36:01.115299 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d09d0c4-2f73-4254-8bcb-b04fe858333d service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:36:01.133622 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 732/1470] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:36:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:36:10.785870 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=22 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:36:11.131803 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=66 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:36:20.828096 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fa60c3-e09e-4f57-8b65-9a9076e42436 None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:36:20.828665 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fa60c3-e09e-4f57-8b65-9a9076e42436 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:36:20.828665 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fa60c3-e09e-4f57-8b65-9a9076e42436 None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:36:20.830036 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 739/1471] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:36:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:36:20.835621 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2949edb-957a-486a-a6cc-f4505e6b3f2f None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:36:20.836022 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2949edb-957a-486a-a6cc-f4505e6b3f2f None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:36:20.836162 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2949edb-957a-486a-a6cc-f4505e6b3f2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:36:20.901007 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-b2949edb-957a-486a-a6cc-f4505e6b3f2f None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:36:20.902207 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-b2949edb-957a-486a-a6cc-f4505e6b3f2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:36:20.997849 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 733/1472] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:36:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 08:36:21.086666 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed5f575-c0d6-445e-83eb-b609a151cc8e None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:36:21.086666 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed5f575-c0d6-445e-83eb-b609a151cc8e None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:36:21.086666 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed5f575-c0d6-445e-83eb-b609a151cc8e None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:36:21.086666 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 740/1473] 10.222.0.57 () {54 vars in 967 bytes} [Wed Aug 10 08:36:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:36:21.088089 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff0a576-2fae-46fb-bc33-2fd63b20b4c5 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:36:21.088376 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff0a576-2fae-46fb-bc33-2fd63b20b4c5 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:36:21.088645 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff0a576-2fae-46fb-bc33-2fd63b20b4c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:36:21.167395 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-aff0a576-2fae-46fb-bc33-2fd63b20b4c5 None None] MFA Rules not processed for user `3263af99fc2f48e79fd1e5b8a1e555a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:36:21.168615 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-aff0a576-2fae-46fb-bc33-2fd63b20b4c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:36:21.189605 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:36:21.246006 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 734/1474] 10.222.0.57 () {58 vars in 1064 bytes} [Wed Aug 10 08:36:21 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:36:21.254011 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed5f575-c0d6-445e-83eb-b609a151cc8e None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:36:21.255512 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-2ed5f575-c0d6-445e-83eb-b609a151cc8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:36:21.358433 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed5f575-c0d6-445e-83eb-b609a151cc8e None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:36:21.434187 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89231a34-2b11-4f5b-b54e-ea5ef599e2a2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3263af99fc2f48e79fd1e5b8a1e555a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:36:21.434813 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89231a34-2b11-4f5b-b54e-ea5ef599e2a2 service glance] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:36:21.435090 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89231a34-2b11-4f5b-b54e-ea5ef599e2a2 service glance] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:36:21.435383 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89231a34-2b11-4f5b-b54e-ea5ef599e2a2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:36:21.436997 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-89231a34-2b11-4f5b-b54e-ea5ef599e2a2 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:36:21.491110 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:36:21.545271 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89231a34-2b11-4f5b-b54e-ea5ef599e2a2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:36:21.546698 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89231a34-2b11-4f5b-b54e-ea5ef599e2a2 service glance] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:36:21.565142 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 741/1475] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:36:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:36:23.649109 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0a92dc-7b6c-43fb-975e-3713e73975f6 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:36:23.649673 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0a92dc-7b6c-43fb-975e-3713e73975f6 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:36:23.649673 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0a92dc-7b6c-43fb-975e-3713e73975f6 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:36:23.651045 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 735/1476] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:36:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:36:23.656196 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e53e36b-8cfe-432a-af73-da68da9b67e6 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:36:23.656597 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e53e36b-8cfe-432a-af73-da68da9b67e6 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:36:23.656692 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e53e36b-8cfe-432a-af73-da68da9b67e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:36:23.721324 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-4e53e36b-8cfe-432a-af73-da68da9b67e6 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:36:23.722503 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-4e53e36b-8cfe-432a-af73-da68da9b67e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:36:23.816303 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 742/1477] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:36:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:36:23.840008 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0a92dc-7b6c-43fb-975e-3713e73975f6 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:36:23.841575 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-4c0a92dc-7b6c-43fb-975e-3713e73975f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:36:23.966748 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0a92dc-7b6c-43fb-975e-3713e73975f6 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:36:24.014636 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:36:24.058111 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c2c2d4-e6e3-4c39-b8ca-fd9e14abcbc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:36:24.058708 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c2c2d4-e6e3-4c39-b8ca-fd9e14abcbc1 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:36:24.058980 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c2c2d4-e6e3-4c39-b8ca-fd9e14abcbc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:36:24.059243 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c2c2d4-e6e3-4c39-b8ca-fd9e14abcbc1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:36:24.059890 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c2c2d4-e6e3-4c39-b8ca-fd9e14abcbc1 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:36:24.061637 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: WARNING py.warnings [None req-c1c2c2d4-e6e3-4c39-b8ca-fd9e14abcbc1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 08:36:24.061637 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: warnings.warn(msg) Aug 10 08:36:24.061637 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: Aug 10 08:36:24.062771 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c2c2d4-e6e3-4c39-b8ca-fd9e14abcbc1 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:36:24.081307 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 736/1478] 10.222.0.57 () {56 vars in 1108 bytes} [Wed Aug 10 08:36:23 2022] GET /identity/v3/projects => generated 2553 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:36:26.078641 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b42b7b-cfca-4673-a7c4-d7d60a76c4e0 None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:36:26.079198 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b42b7b-cfca-4673-a7c4-d7d60a76c4e0 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:36:26.079198 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b42b7b-cfca-4673-a7c4-d7d60a76c4e0 None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:36:26.080514 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 743/1479] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:36:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:36:26.086336 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bf075f-c3d7-40ae-ab16-a76e8f922426 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:36:26.086781 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bf075f-c3d7-40ae-ab16-a76e8f922426 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:36:26.086887 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bf075f-c3d7-40ae-ab16-a76e8f922426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:36:26.152007 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-40bf075f-c3d7-40ae-ab16-a76e8f922426 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:36:26.153166 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-40bf075f-c3d7-40ae-ab16-a76e8f922426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:36:26.249889 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 737/1480] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:36:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:36:26.292850 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b42b7b-cfca-4673-a7c4-d7d60a76c4e0 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:36:26.294486 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-09b42b7b-cfca-4673-a7c4-d7d60a76c4e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:36:26.304948 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:36:26.439933 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b42b7b-cfca-4673-a7c4-d7d60a76c4e0 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:36:26.520234 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:36:26.553273 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f4cf50-5525-4d96-9e3f-536fcb1a2389 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a943008193914f76a663d489bd73e873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:36:26.553867 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f4cf50-5525-4d96-9e3f-536fcb1a2389 service nova] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:36:26.554145 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f4cf50-5525-4d96-9e3f-536fcb1a2389 service nova] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:36:26.554414 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f4cf50-5525-4d96-9e3f-536fcb1a2389 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:36:26.555995 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-41f4cf50-5525-4d96-9e3f-536fcb1a2389 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:36:26.642196 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f4cf50-5525-4d96-9e3f-536fcb1a2389 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:36:26.643769 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f4cf50-5525-4d96-9e3f-536fcb1a2389 service nova] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:36:26.662070 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 744/1481] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:36:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:36:28.880051 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bc4efa-911a-4f7b-8e2b-a17144c4c436 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:36:28.880595 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bc4efa-911a-4f7b-8e2b-a17144c4c436 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:36:28.880896 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bc4efa-911a-4f7b-8e2b-a17144c4c436 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:36:28.882779 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 738/1482] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:36:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:36:28.887953 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c926f4d9-de1f-48af-8148-1705a101bdc2 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:36:28.888356 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c926f4d9-de1f-48af-8148-1705a101bdc2 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:36:28.888493 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c926f4d9-de1f-48af-8148-1705a101bdc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:36:28.952481 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-c926f4d9-de1f-48af-8148-1705a101bdc2 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:36:28.953692 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-c926f4d9-de1f-48af-8148-1705a101bdc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:36:28.998864 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:36:29.048774 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 745/1483] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:36:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 08:36:29.086263 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bc4efa-911a-4f7b-8e2b-a17144c4c436 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:36:29.087800 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-54bc4efa-911a-4f7b-8e2b-a17144c4c436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:36:29.110705 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:36:29.191555 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bc4efa-911a-4f7b-8e2b-a17144c4c436 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:36:29.265072 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca58bc59-b809-40a4-9e21-1d110557fe9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:36:29.265676 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca58bc59-b809-40a4-9e21-1d110557fe9d service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:36:29.265955 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca58bc59-b809-40a4-9e21-1d110557fe9d service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:36:29.266222 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca58bc59-b809-40a4-9e21-1d110557fe9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:36:29.267852 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-ca58bc59-b809-40a4-9e21-1d110557fe9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:36:29.340608 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:36:29.374980 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca58bc59-b809-40a4-9e21-1d110557fe9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:36:29.376425 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca58bc59-b809-40a4-9e21-1d110557fe9d service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:36:29.395529 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 739/1484] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:36:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:36:31.461370 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0f31db-5b2c-4be2-afb5-80205c78bb82 None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:36:31.461961 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0f31db-5b2c-4be2-afb5-80205c78bb82 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:36:31.461961 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0f31db-5b2c-4be2-afb5-80205c78bb82 None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:36:31.463303 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 746/1485] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:36:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:36:31.468367 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f423bfae-1192-4634-b6fa-74cf14bbc818 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:36:31.468830 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f423bfae-1192-4634-b6fa-74cf14bbc818 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:36:31.468972 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f423bfae-1192-4634-b6fa-74cf14bbc818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:36:31.533890 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-f423bfae-1192-4634-b6fa-74cf14bbc818 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:36:31.535027 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-f423bfae-1192-4634-b6fa-74cf14bbc818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:36:31.631862 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 740/1486] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:36:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:36:31.670541 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0f31db-5b2c-4be2-afb5-80205c78bb82 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:36:31.672053 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-6f0f31db-5b2c-4be2-afb5-80205c78bb82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:36:31.774945 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0f31db-5b2c-4be2-afb5-80205c78bb82 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:36:31.845592 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:36:31.847375 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aa0bb2-d6bc-46d1-a8db-8a6d1f2e036c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:36:31.847978 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aa0bb2-d6bc-46d1-a8db-8a6d1f2e036c service neutron] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:36:31.848259 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aa0bb2-d6bc-46d1-a8db-8a6d1f2e036c service neutron] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:36:31.848531 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aa0bb2-d6bc-46d1-a8db-8a6d1f2e036c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:36:31.850238 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-c0aa0bb2-d6bc-46d1-a8db-8a6d1f2e036c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:36:31.959365 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0aa0bb2-d6bc-46d1-a8db-8a6d1f2e036c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:36:31.960807 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0aa0bb2-d6bc-46d1-a8db-8a6d1f2e036c service neutron] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:36:31.978508 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 747/1487] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:36:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:36:34.155062 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7576c428-b9f1-4fe1-b29c-c7ae921716ee None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:36:34.155612 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7576c428-b9f1-4fe1-b29c-c7ae921716ee None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:36:34.155612 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7576c428-b9f1-4fe1-b29c-c7ae921716ee None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:36:34.157046 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 741/1488] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:36:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:36:34.163071 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0f3ddb-fe08-495c-92ec-b7ac629e13b9 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:36:34.163543 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0f3ddb-fe08-495c-92ec-b7ac629e13b9 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:36:34.163648 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0f3ddb-fe08-495c-92ec-b7ac629e13b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:36:34.229302 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-ff0f3ddb-fe08-495c-92ec-b7ac629e13b9 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:36:34.230580 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-ff0f3ddb-fe08-495c-92ec-b7ac629e13b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:36:34.286520 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:36:34.331143 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 748/1489] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:36:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:36:34.373886 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7576c428-b9f1-4fe1-b29c-c7ae921716ee None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:36:34.375473 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-7576c428-b9f1-4fe1-b29c-c7ae921716ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:36:34.413246 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:36:34.482067 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7576c428-b9f1-4fe1-b29c-c7ae921716ee None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:36:34.556558 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45543640-99ff-43ae-86fb-15f7b7a5996f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:36:34.557183 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45543640-99ff-43ae-86fb-15f7b7a5996f service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:36:34.557455 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45543640-99ff-43ae-86fb-15f7b7a5996f service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:36:34.557718 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45543640-99ff-43ae-86fb-15f7b7a5996f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:36:34.559294 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-45543640-99ff-43ae-86fb-15f7b7a5996f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:36:34.637955 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:36:34.666768 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45543640-99ff-43ae-86fb-15f7b7a5996f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:36:34.668270 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45543640-99ff-43ae-86fb-15f7b7a5996f service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:36:34.686800 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 742/1490] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:36:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:36:37.114537 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bcfa8f-09e4-4fdd-bbfa-71fa07ee3a77 None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:36:37.115145 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bcfa8f-09e4-4fdd-bbfa-71fa07ee3a77 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:36:37.115256 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bcfa8f-09e4-4fdd-bbfa-71fa07ee3a77 None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:36:37.116655 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 749/1491] 10.222.0.57 () {54 vars in 927 bytes} [Wed Aug 10 08:36:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:36:37.121866 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89295679-35d5-463f-8f2e-19e9233d5980 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:36:37.122305 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89295679-35d5-463f-8f2e-19e9233d5980 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:36:37.122418 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89295679-35d5-463f-8f2e-19e9233d5980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:36:37.188893 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-89295679-35d5-463f-8f2e-19e9233d5980 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:36:37.190117 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-89295679-35d5-463f-8f2e-19e9233d5980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:36:37.288547 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 743/1492] 10.222.0.57 () {58 vars in 1024 bytes} [Wed Aug 10 08:36:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:36:37.329015 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bcfa8f-09e4-4fdd-bbfa-71fa07ee3a77 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:36:37.330574 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-68bcfa8f-09e4-4fdd-bbfa-71fa07ee3a77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:36:37.435966 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bcfa8f-09e4-4fdd-bbfa-71fa07ee3a77 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:36:37.508575 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6421dddc-c11b-4255-83ca-b3227407a317 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:36:37.509207 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6421dddc-c11b-4255-83ca-b3227407a317 service neutron] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:36:37.509487 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6421dddc-c11b-4255-83ca-b3227407a317 service neutron] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:36:37.509759 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6421dddc-c11b-4255-83ca-b3227407a317 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:36:37.511358 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-6421dddc-c11b-4255-83ca-b3227407a317 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:36:37.520311 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:36:37.618753 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6421dddc-c11b-4255-83ca-b3227407a317 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:36:37.620169 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6421dddc-c11b-4255-83ca-b3227407a317 service neutron] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:36:37.639992 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 750/1493] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:36:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:36:47.285338 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=82 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:36:47.637795 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=50 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:38:00.827325 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d594a149-e955-4176-a0b5-bac8daef7370 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:38:00.827923 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d594a149-e955-4176-a0b5-bac8daef7370 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:38:00.827923 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d594a149-e955-4176-a0b5-bac8daef7370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:38:00.903915 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-d594a149-e955-4176-a0b5-bac8daef7370 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:38:00.905155 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-d594a149-e955-4176-a0b5-bac8daef7370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:38:01.002261 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 744/1494] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:38:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:38:01.011483 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de8f6f5-3e39-4b82-aed7-096dbca1dcdb None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:38:01.012007 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de8f6f5-3e39-4b82-aed7-096dbca1dcdb None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:38:01.012300 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de8f6f5-3e39-4b82-aed7-096dbca1dcdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:38:01.087428 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-9de8f6f5-3e39-4b82-aed7-096dbca1dcdb None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:38:01.088559 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-9de8f6f5-3e39-4b82-aed7-096dbca1dcdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:38:01.183432 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 751/1495] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:38:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:38:01.188241 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d594a149-e955-4176-a0b5-bac8daef7370 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:38:01.189480 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-d594a149-e955-4176-a0b5-bac8daef7370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:38:01.201252 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d594a149-e955-4176-a0b5-bac8daef7370 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:38:01.264659 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:38:01.299457 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adc2148-2b60-4bb2-8e7c-aeedb0ac4d7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:38:01.300052 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adc2148-2b60-4bb2-8e7c-aeedb0ac4d7b admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:38:01.300324 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adc2148-2b60-4bb2-8e7c-aeedb0ac4d7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:38:01.300588 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adc2148-2b60-4bb2-8e7c-aeedb0ac4d7b admin admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:38:01.301883 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adc2148-2b60-4bb2-8e7c-aeedb0ac4d7b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:38:01.302187 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adc2148-2b60-4bb2-8e7c-aeedb0ac4d7b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:38:01.303232 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: WARNING py.warnings [None req-8adc2148-2b60-4bb2-8e7c-aeedb0ac4d7b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 10 08:38:01.303232 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: warnings.warn(msg) Aug 10 08:38:01.303232 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: Aug 10 08:38:01.304261 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adc2148-2b60-4bb2-8e7c-aeedb0ac4d7b admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:38:01.315714 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 745/1496] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Aug 10 08:38:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:38:01.324019 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de8f6f5-3e39-4b82-aed7-096dbca1dcdb None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:38:01.333278 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de8f6f5-3e39-4b82-aed7-096dbca1dcdb None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:38:01.394081 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:38:01.429076 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab06667-d1e3-4145-9e37-315cdb716f34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:38:01.429681 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab06667-d1e3-4145-9e37-315cdb716f34 admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:38:01.429955 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab06667-d1e3-4145-9e37-315cdb716f34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:38:01.430221 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab06667-d1e3-4145-9e37-315cdb716f34 admin admin] PATH_INFO: `/v3/projects` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:38:01.430834 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab06667-d1e3-4145-9e37-315cdb716f34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:38:01.432548 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: WARNING py.warnings [None req-8ab06667-d1e3-4145-9e37-315cdb716f34 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 08:38:01.432548 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: warnings.warn(msg) Aug 10 08:38:01.432548 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: Aug 10 08:38:01.433779 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab06667-d1e3-4145-9e37-315cdb716f34 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:38:01.566963 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 752/1497] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 08:38:01 2022] POST /identity/v3/projects => generated 379 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:38:01.571295 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adc2148-2b60-4bb2-8e7c-aeedb0ac4d7b admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:38:01.572957 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-8adc2148-2b60-4bb2-8e7c-aeedb0ac4d7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:38:01.606370 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adc2148-2b60-4bb2-8e7c-aeedb0ac4d7b admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:38:01.699992 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2456c3c5-30da-4db3-ab3f-35b7fbe45c1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:38:01.700586 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2456c3c5-30da-4db3-ab3f-35b7fbe45c1f admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:38:01.700859 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2456c3c5-30da-4db3-ab3f-35b7fbe45c1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:38:01.701152 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2456c3c5-30da-4db3-ab3f-35b7fbe45c1f admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:38:01.701783 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2456c3c5-30da-4db3-ab3f-35b7fbe45c1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:38:01.702771 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: WARNING py.warnings [None req-2456c3c5-30da-4db3-ab3f-35b7fbe45c1f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 08:38:01.702771 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: warnings.warn(msg) Aug 10 08:38:01.702771 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: Aug 10 08:38:01.703973 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2456c3c5-30da-4db3-ab3f-35b7fbe45c1f admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:38:01.816990 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 746/1498] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 08:38:01 2022] POST /identity/v3/users => generated 290 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:38:01.821172 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab06667-d1e3-4145-9e37-315cdb716f34 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:38:01.822708 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-8ab06667-d1e3-4145-9e37-315cdb716f34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:38:01.884304 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab06667-d1e3-4145-9e37-315cdb716f34 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:38:01.967617 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:38:01.978160 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db94486e-fc29-4ea1-9144-9c5be5dc88da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:38:01.981218 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db94486e-fc29-4ea1-9144-9c5be5dc88da admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:38:01.981218 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db94486e-fc29-4ea1-9144-9c5be5dc88da admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:38:01.981218 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db94486e-fc29-4ea1-9144-9c5be5dc88da admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:38:01.981218 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db94486e-fc29-4ea1-9144-9c5be5dc88da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:38:01.981218 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: WARNING py.warnings [None req-db94486e-fc29-4ea1-9144-9c5be5dc88da admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 10 08:38:01.981218 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: warnings.warn(msg) Aug 10 08:38:01.981218 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: Aug 10 08:38:01.981805 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db94486e-fc29-4ea1-9144-9c5be5dc88da admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:38:01.993735 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 753/1499] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:38:01 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:38:01.997868 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2456c3c5-30da-4db3-ab3f-35b7fbe45c1f admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:38:01.999394 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-2456c3c5-30da-4db3-ab3f-35b7fbe45c1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:38:02.011731 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2456c3c5-30da-4db3-ab3f-35b7fbe45c1f admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:38:02.037328 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:38:02.106189 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4937346c-d8e6-4936-8a70-345cbfcb6f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:38:02.106796 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4937346c-d8e6-4936-8a70-345cbfcb6f39 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:38:02.107074 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4937346c-d8e6-4936-8a70-345cbfcb6f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:38:02.107344 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4937346c-d8e6-4936-8a70-345cbfcb6f39 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:38:02.107983 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4937346c-d8e6-4936-8a70-345cbfcb6f39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:38:02.109052 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: WARNING py.warnings [None req-4937346c-d8e6-4936-8a70-345cbfcb6f39 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 10 08:38:02.109052 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: warnings.warn(msg) Aug 10 08:38:02.109052 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: Aug 10 08:38:02.109947 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4937346c-d8e6-4936-8a70-345cbfcb6f39 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:38:02.122830 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 747/1500] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:38:01 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:38:02.128446 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db94486e-fc29-4ea1-9144-9c5be5dc88da admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:38:02.131208 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-db94486e-fc29-4ea1-9144-9c5be5dc88da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:38:02.165965 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db94486e-fc29-4ea1-9144-9c5be5dc88da admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:38:02.261248 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9f1277-5c0d-4ee0-846f-3f86ccc543c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:38:02.261824 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9f1277-5c0d-4ee0-846f-3f86ccc543c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:38:02.262104 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9f1277-5c0d-4ee0-846f-3f86ccc543c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:38:02.262375 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9f1277-5c0d-4ee0-846f-3f86ccc543c1 admin admin] PATH_INFO: `/v3/projects/b871f0922e0449528f08f436bf85239b/users/47808abfc3a843a5898976657be6d483/roles/44b64db484f24a46bf0b7ced46b519bf` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:38:02.285050 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9f1277-5c0d-4ee0-846f-3f86ccc543c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b871f0922e0449528f08f436bf85239b, user_id=47808abfc3a843a5898976657be6d483, role_id=44b64db484f24a46bf0b7ced46b519bf)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:38:02.286325 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: WARNING py.warnings [None req-af9f1277-5c0d-4ee0-846f-3f86ccc543c1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 08:38:02.286325 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: warnings.warn(msg) Aug 10 08:38:02.286325 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: Aug 10 08:38:02.288301 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9f1277-5c0d-4ee0-846f-3f86ccc543c1 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:38:02.397568 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 754/1501] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 08:38:02 2022] PUT /identity/v3/projects/b871f0922e0449528f08f436bf85239b/users/47808abfc3a843a5898976657be6d483/roles/44b64db484f24a46bf0b7ced46b519bf => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:38:02.411199 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b56eeb8-5293-4325-972d-1f8cfc34f7f8 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:38:02.411488 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b56eeb8-5293-4325-972d-1f8cfc34f7f8 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:38:02.411764 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b56eeb8-5293-4325-972d-1f8cfc34f7f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:38:02.467841 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-4b56eeb8-5293-4325-972d-1f8cfc34f7f8 None None] MFA Rules not processed for user `47808abfc3a843a5898976657be6d483`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:38:02.469075 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-4b56eeb8-5293-4325-972d-1f8cfc34f7f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:38:02.546278 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 748/1502] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:38:02 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:38:02.561855 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5237050-99ba-4eba-be89-350ade439202 None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:38:02.562140 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5237050-99ba-4eba-be89-350ade439202 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:38:02.562408 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5237050-99ba-4eba-be89-350ade439202 None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:38:02.563829 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 755/1503] 10.222.0.57 () {54 vars in 964 bytes} [Wed Aug 10 08:38:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:38:02.569670 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf21296b-497c-4b7a-b2a3-d091dc778358 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:38:02.569996 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf21296b-497c-4b7a-b2a3-d091dc778358 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:38:02.570335 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf21296b-497c-4b7a-b2a3-d091dc778358 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:38:02.600842 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:38:02.645413 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-cf21296b-497c-4b7a-b2a3-d091dc778358 None None] MFA Rules not processed for user `f1ba74e28ced4cbba857f54222dfbe41`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:38:02.647187 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-cf21296b-497c-4b7a-b2a3-d091dc778358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:38:02.721791 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 749/1504] 10.222.0.57 () {58 vars in 1061 bytes} [Wed Aug 10 08:38:02 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:38:02.730516 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5237050-99ba-4eba-be89-350ade439202 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:38:02.732024 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-f5237050-99ba-4eba-be89-350ade439202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:38:02.781885 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:38:02.836681 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5237050-99ba-4eba-be89-350ade439202 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:38:02.909793 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713764e0-f2b4-4cae-a7af-d1553ef651ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ba74e28ced4cbba857f54222dfbe41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:38:02.910607 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713764e0-f2b4-4cae-a7af-d1553ef651ae service cinder] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:38:02.911249 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713764e0-f2b4-4cae-a7af-d1553ef651ae service cinder] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:38:02.911528 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713764e0-f2b4-4cae-a7af-d1553ef651ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:38:02.913134 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-713764e0-f2b4-4cae-a7af-d1553ef651ae service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:38:03.011006 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713764e0-f2b4-4cae-a7af-d1553ef651ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:38:03.012458 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713764e0-f2b4-4cae-a7af-d1553ef651ae service cinder] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:38:03.015889 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:38:03.031178 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 756/1505] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:38:02 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:38:03.075781 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38968af9-d651-4fb8-8ec2-7013fbbd0326 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:38:03.076191 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38968af9-d651-4fb8-8ec2-7013fbbd0326 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:38:03.076324 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38968af9-d651-4fb8-8ec2-7013fbbd0326 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:38:03.077621 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 750/1506] 10.222.0.57 () {52 vars in 830 bytes} [Wed Aug 10 08:38:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:38:03.081167 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713764e0-f2b4-4cae-a7af-d1553ef651ae service cinder] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:38:03.082984 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-713764e0-f2b4-4cae-a7af-d1553ef651ae service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:38:03.182241 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713764e0-f2b4-4cae-a7af-d1553ef651ae service cinder] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:38:03.276074 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4db03a-39e9-404f-8d15-489ae3fe8ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:38:03.276688 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4db03a-39e9-404f-8d15-489ae3fe8ae4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:38:03.277053 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4db03a-39e9-404f-8d15-489ae3fe8ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:38:03.277323 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4db03a-39e9-404f-8d15-489ae3fe8ae4 admin admin] PATH_INFO: `/v3/users/47808abfc3a843a5898976657be6d483` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:38:03.290242 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4db03a-39e9-404f-8d15-489ae3fe8ae4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47808abfc3a843a5898976657be6d483)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:38:03.291249 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: WARNING py.warnings [None req-df4db03a-39e9-404f-8d15-489ae3fe8ae4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 08:38:03.291249 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: warnings.warn(msg) Aug 10 08:38:03.291249 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: Aug 10 08:38:03.292455 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4db03a-39e9-404f-8d15-489ae3fe8ae4 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:38:03.301756 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:38:03.494730 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-df4db03a-39e9-404f-8d15-489ae3fe8ae4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47808abfc3a843a5898976657be6d483'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:38:03.518193 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-df4db03a-39e9-404f-8d15-489ae3fe8ae4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47808abfc3a843a5898976657be6d483'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:38:03.532881 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-df4db03a-39e9-404f-8d15-489ae3fe8ae4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47808abfc3a843a5898976657be6d483'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:38:03.610947 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-df4db03a-39e9-404f-8d15-489ae3fe8ae4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47808abfc3a843a5898976657be6d483'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:38:03.611208 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-df4db03a-39e9-404f-8d15-489ae3fe8ae4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47808abfc3a843a5898976657be6d483'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:38:03.617395 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 757/1507] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 08:38:03 2022] DELETE /identity/v3/users/47808abfc3a843a5898976657be6d483 => generated 0 bytes in 537 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:38:03.621712 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08beed9-c51a-4b25-9037-57d5f588c805 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:38:03.622035 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08beed9-c51a-4b25-9037-57d5f588c805 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:38:03.622307 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08beed9-c51a-4b25-9037-57d5f588c805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:38:03.679111 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-d08beed9-c51a-4b25-9037-57d5f588c805 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:38:03.680311 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-d08beed9-c51a-4b25-9037-57d5f588c805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:38:03.753048 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:38:03.765411 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 751/1508] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:38:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:38:03.775691 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4db03a-39e9-404f-8d15-489ae3fe8ae4 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:38:03.776936 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-df4db03a-39e9-404f-8d15-489ae3fe8ae4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:38:03.874478 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4db03a-39e9-404f-8d15-489ae3fe8ae4 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:38:03.929121 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:38:03.948382 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b23aa0-dfe4-49a8-bd8e-e35be9536f08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:38:03.949022 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b23aa0-dfe4-49a8-bd8e-e35be9536f08 service neutron] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:38:03.949303 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b23aa0-dfe4-49a8-bd8e-e35be9536f08 service neutron] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:38:03.949570 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b23aa0-dfe4-49a8-bd8e-e35be9536f08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:38:03.951149 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-29b23aa0-dfe4-49a8-bd8e-e35be9536f08 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:38:04.069258 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b23aa0-dfe4-49a8-bd8e-e35be9536f08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:38:04.070624 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b23aa0-dfe4-49a8-bd8e-e35be9536f08 service neutron] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:38:04.089012 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 758/1509] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:38:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:38:04.575475 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08beed9-c51a-4b25-9037-57d5f588c805 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:38:04.576704 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-d08beed9-c51a-4b25-9037-57d5f588c805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:38:04.588080 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08beed9-c51a-4b25-9037-57d5f588c805 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:38:04.689899 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59ed864-0c40-416b-8f1f-23d7b6a9d6b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:38:04.690531 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59ed864-0c40-416b-8f1f-23d7b6a9d6b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:38:04.690807 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59ed864-0c40-416b-8f1f-23d7b6a9d6b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:38:04.691073 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59ed864-0c40-416b-8f1f-23d7b6a9d6b3 admin admin] PATH_INFO: `/v3/projects/b871f0922e0449528f08f436bf85239b` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:38:04.701526 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59ed864-0c40-416b-8f1f-23d7b6a9d6b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b871f0922e0449528f08f436bf85239b)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:38:04.702538 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: WARNING py.warnings [None req-a59ed864-0c40-416b-8f1f-23d7b6a9d6b3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 08:38:04.702538 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: warnings.warn(msg) Aug 10 08:38:04.702538 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: Aug 10 08:38:04.703746 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59ed864-0c40-416b-8f1f-23d7b6a9d6b3 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:38:04.814844 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-a59ed864-0c40-416b-8f1f-23d7b6a9d6b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b871f0922e0449528f08f436bf85239b', 'request_id': 'req-a59ed864-0c40-416b-8f1f-23d7b6a9d6b3'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:38:04.881906 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-a59ed864-0c40-416b-8f1f-23d7b6a9d6b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b871f0922e0449528f08f436bf85239b', 'request_id': 'req-a59ed864-0c40-416b-8f1f-23d7b6a9d6b3'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:38:04.888122 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-a59ed864-0c40-416b-8f1f-23d7b6a9d6b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b871f0922e0449528f08f436bf85239b', 'request_id': 'req-a59ed864-0c40-416b-8f1f-23d7b6a9d6b3'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:38:04.918927 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-a59ed864-0c40-416b-8f1f-23d7b6a9d6b3 admin admin] The token cache is being invalidate because project b871f0922e0449528f08f436bf85239b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 08:38:04.919215 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-a59ed864-0c40-416b-8f1f-23d7b6a9d6b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:38:04.919641 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-a59ed864-0c40-416b-8f1f-23d7b6a9d6b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:38:04.921144 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 752/1510] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 08:38:04 2022] DELETE /identity/v3/projects/b871f0922e0449528f08f436bf85239b => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:38:14.086710 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=62 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:38:14.916718 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:39:31.446024 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62803cde-7847-4bb1-a1f3-b4807add6b35 None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:39:31.446550 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62803cde-7847-4bb1-a1f3-b4807add6b35 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:39:31.446550 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62803cde-7847-4bb1-a1f3-b4807add6b35 None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:39:31.447904 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 759/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 08:39:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:39:31.452752 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680afc70-c9f6-4ddd-9009-f6b58a417499 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:39:31.453235 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680afc70-c9f6-4ddd-9009-f6b58a417499 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:39:31.453345 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680afc70-c9f6-4ddd-9009-f6b58a417499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:39:31.532155 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-680afc70-c9f6-4ddd-9009-f6b58a417499 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:39:31.533320 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-680afc70-c9f6-4ddd-9009-f6b58a417499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:39:31.634863 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 753/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 08:39:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:39:31.787875 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62803cde-7847-4bb1-a1f3-b4807add6b35 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:39:31.789988 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-62803cde-7847-4bb1-a1f3-b4807add6b35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:39:31.909486 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62803cde-7847-4bb1-a1f3-b4807add6b35 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:39:31.915558 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fd6fe3-8ebc-4b72-86c1-8a82e4812521 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a943008193914f76a663d489bd73e873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:39:31.916157 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fd6fe3-8ebc-4b72-86c1-8a82e4812521 service nova] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:39:31.916437 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fd6fe3-8ebc-4b72-86c1-8a82e4812521 service nova] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:39:31.916706 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fd6fe3-8ebc-4b72-86c1-8a82e4812521 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:39:31.924610 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fd6fe3-8ebc-4b72-86c1-8a82e4812521 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:39:31.926395 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fd6fe3-8ebc-4b72-86c1-8a82e4812521 service nova] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:39:31.935657 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 760/1513] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:39:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:39:32.769565 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd421dc9-5e07-4f1b-9593-e9d94b39b79a None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:39:32.770797 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd421dc9-5e07-4f1b-9593-e9d94b39b79a None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:39:32.771080 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd421dc9-5e07-4f1b-9593-e9d94b39b79a None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:39:32.772439 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 754/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 08:39:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 08:39:32.777158 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefc8ff2-523c-424e-aae6-5d11cfc51f08 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:39:32.777631 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefc8ff2-523c-424e-aae6-5d11cfc51f08 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:39:32.777742 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefc8ff2-523c-424e-aae6-5d11cfc51f08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:39:32.796622 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-fefc8ff2-523c-424e-aae6-5d11cfc51f08 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:39:32.797845 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-fefc8ff2-523c-424e-aae6-5d11cfc51f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:39:32.806648 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 761/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 08:39:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:39:32.958712 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd421dc9-5e07-4f1b-9593-e9d94b39b79a None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:39:32.973462 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd421dc9-5e07-4f1b-9593-e9d94b39b79a None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:39:32.978814 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6d5633-65f6-4e57-8d33-9ca616b245b4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a943008193914f76a663d489bd73e873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:39:32.979403 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6d5633-65f6-4e57-8d33-9ca616b245b4 service nova] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:39:32.979685 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6d5633-65f6-4e57-8d33-9ca616b245b4 service nova] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:39:32.979956 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6d5633-65f6-4e57-8d33-9ca616b245b4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:39:32.987858 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6d5633-65f6-4e57-8d33-9ca616b245b4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:39:32.989587 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6d5633-65f6-4e57-8d33-9ca616b245b4 service nova] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:39:32.998793 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 755/1516] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:39:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:39:33.168828 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef9b8a6-9e7e-4207-a4fe-3ddfcf3712a3 None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:39:33.169165 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef9b8a6-9e7e-4207-a4fe-3ddfcf3712a3 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:39:33.169441 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef9b8a6-9e7e-4207-a4fe-3ddfcf3712a3 None None] PATH_INFO: `/v3` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:39:33.170768 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 762/1517] 10.222.0.59 () {54 vars in 925 bytes} [Wed Aug 10 08:39:33 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:39:33.177135 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e6b988-0525-46d9-b665-ecbc0485fae7 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:39:33.177425 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e6b988-0525-46d9-b665-ecbc0485fae7 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:39:33.177699 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e6b988-0525-46d9-b665-ecbc0485fae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:39:33.236053 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-f7e6b988-0525-46d9-b665-ecbc0485fae7 None None] MFA Rules not processed for user `2912307c56594125b157f1cfdf7c96a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:39:33.237301 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-f7e6b988-0525-46d9-b665-ecbc0485fae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:39:33.254051 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:39:33.309221 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 756/1518] 10.222.0.59 () {58 vars in 1015 bytes} [Wed Aug 10 08:39:33 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:39:42.659652 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45d882a-020f-40da-ba36-888d09db8d8e None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:39:42.660752 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45d882a-020f-40da-ba36-888d09db8d8e None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:39:42.660752 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45d882a-020f-40da-ba36-888d09db8d8e None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:39:42.662119 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 763/1519] 10.222.0.57 () {54 vars in 961 bytes} [Wed Aug 10 08:39:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:39:42.667029 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a2fc0b-257a-4e5a-8307-f24017d9b9a9 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:39:42.667439 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a2fc0b-257a-4e5a-8307-f24017d9b9a9 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:39:42.667566 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a2fc0b-257a-4e5a-8307-f24017d9b9a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:39:42.686479 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-04a2fc0b-257a-4e5a-8307-f24017d9b9a9 None None] MFA Rules not processed for user `2912307c56594125b157f1cfdf7c96a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:39:42.687672 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-04a2fc0b-257a-4e5a-8307-f24017d9b9a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:39:42.696305 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 757/1520] 10.222.0.57 () {58 vars in 1058 bytes} [Wed Aug 10 08:39:42 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:39:42.703628 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45d882a-020f-40da-ba36-888d09db8d8e None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:39:42.717216 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45d882a-020f-40da-ba36-888d09db8d8e None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:39:42.722059 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d247580-20b7-495f-9a6d-bb3f8c1d02ad service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2912307c56594125b157f1cfdf7c96a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:39:42.723215 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d247580-20b7-495f-9a6d-bb3f8c1d02ad service placement] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:39:42.723478 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d247580-20b7-495f-9a6d-bb3f8c1d02ad service placement] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:39:42.723745 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d247580-20b7-495f-9a6d-bb3f8c1d02ad service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:39:42.731906 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d247580-20b7-495f-9a6d-bb3f8c1d02ad service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:39:42.733511 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d247580-20b7-495f-9a6d-bb3f8c1d02ad service placement] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:39:42.743060 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 764/1521] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:39:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:39:44.272903 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f50dde-599d-40fc-aba3-7462d82dc388 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:39:44.279409 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f50dde-599d-40fc-aba3-7462d82dc388 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:39:44.279409 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f50dde-599d-40fc-aba3-7462d82dc388 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:39:44.279409 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 758/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 08:39:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:39:44.279707 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6124d422-4d79-4091-b93f-bea5f6032898 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:39:44.280102 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6124d422-4d79-4091-b93f-bea5f6032898 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:39:44.280102 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6124d422-4d79-4091-b93f-bea5f6032898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:39:44.299666 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-6124d422-4d79-4091-b93f-bea5f6032898 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:39:44.300945 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-6124d422-4d79-4091-b93f-bea5f6032898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:39:44.308977 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 765/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 08:39:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:39:44.465037 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f50dde-599d-40fc-aba3-7462d82dc388 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:39:44.479865 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f50dde-599d-40fc-aba3-7462d82dc388 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:39:44.486056 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913c5c47-595d-4b2e-9208-6e1724a91c68 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a943008193914f76a663d489bd73e873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:39:44.486652 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913c5c47-595d-4b2e-9208-6e1724a91c68 service nova] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:39:44.486930 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913c5c47-595d-4b2e-9208-6e1724a91c68 service nova] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:39:44.487198 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913c5c47-595d-4b2e-9208-6e1724a91c68 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:39:44.496101 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913c5c47-595d-4b2e-9208-6e1724a91c68 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:39:44.497709 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913c5c47-595d-4b2e-9208-6e1724a91c68 service nova] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:39:44.512326 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 759/1524] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:39:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:39:45.351830 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e62feb-87bd-47d6-b6dd-63f77cd99415 None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:39:45.352379 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e62feb-87bd-47d6-b6dd-63f77cd99415 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:39:45.352379 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e62feb-87bd-47d6-b6dd-63f77cd99415 None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:39:45.353767 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 766/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 08:39:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:39:45.358506 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87048055-0b40-43f5-8a60-d449444aa60d None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:39:45.358906 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87048055-0b40-43f5-8a60-d449444aa60d None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:39:45.359051 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87048055-0b40-43f5-8a60-d449444aa60d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:39:45.378548 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-87048055-0b40-43f5-8a60-d449444aa60d None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:39:45.380381 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-87048055-0b40-43f5-8a60-d449444aa60d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:39:45.388283 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 760/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 08:39:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:39:45.542038 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e62feb-87bd-47d6-b6dd-63f77cd99415 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:39:45.557374 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e62feb-87bd-47d6-b6dd-63f77cd99415 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:39:45.562674 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d398d0fb-a0ce-4ec1-8e49-e7cb94f665b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a943008193914f76a663d489bd73e873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:39:45.563270 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d398d0fb-a0ce-4ec1-8e49-e7cb94f665b1 service nova] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:39:45.563532 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d398d0fb-a0ce-4ec1-8e49-e7cb94f665b1 service nova] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:39:45.563797 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d398d0fb-a0ce-4ec1-8e49-e7cb94f665b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:39:45.572801 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d398d0fb-a0ce-4ec1-8e49-e7cb94f665b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:39:45.574397 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d398d0fb-a0ce-4ec1-8e49-e7cb94f665b1 service nova] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:39:45.583945 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 767/1527] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:39:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:39:49.561323 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8976e17c-03b3-432b-86da-b3a737d32347 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:39:49.561893 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8976e17c-03b3-432b-86da-b3a737d32347 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:39:49.561893 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8976e17c-03b3-432b-86da-b3a737d32347 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:39:49.563227 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 761/1528] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 08:39:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 08:39:49.569126 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08879202-0432-41e7-bb23-d39ad1fafd12 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:39:49.569557 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08879202-0432-41e7-bb23-d39ad1fafd12 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:39:49.569760 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08879202-0432-41e7-bb23-d39ad1fafd12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:39:49.589448 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-08879202-0432-41e7-bb23-d39ad1fafd12 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:39:49.590639 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-08879202-0432-41e7-bb23-d39ad1fafd12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:39:49.598495 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 768/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 08:39:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:39:49.674525 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8976e17c-03b3-432b-86da-b3a737d32347 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:39:49.676853 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-8976e17c-03b3-432b-86da-b3a737d32347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:39:49.755647 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8976e17c-03b3-432b-86da-b3a737d32347 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:39:49.759383 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dd745d-e879-4b08-a18d-f77d2bace55d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:39:49.759971 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dd745d-e879-4b08-a18d-f77d2bace55d service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:39:49.760242 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dd745d-e879-4b08-a18d-f77d2bace55d service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:39:49.760505 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dd745d-e879-4b08-a18d-f77d2bace55d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:39:49.770717 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dd745d-e879-4b08-a18d-f77d2bace55d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:39:49.772129 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dd745d-e879-4b08-a18d-f77d2bace55d service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:39:49.781457 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 762/1530] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:39:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:39:51.773574 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596c43c3-db3e-473a-aeeb-45bf76ad7877 None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:39:51.774141 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596c43c3-db3e-473a-aeeb-45bf76ad7877 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:39:51.774141 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596c43c3-db3e-473a-aeeb-45bf76ad7877 None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:39:51.775473 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 769/1531] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 08:39:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:39:51.780641 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cdb25f-066a-4f9e-8995-20378bee1bac None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:39:51.781089 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cdb25f-066a-4f9e-8995-20378bee1bac None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:39:51.781234 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cdb25f-066a-4f9e-8995-20378bee1bac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:39:51.801604 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-e0cdb25f-066a-4f9e-8995-20378bee1bac None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:39:51.802797 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-e0cdb25f-066a-4f9e-8995-20378bee1bac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:39:51.810530 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 763/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 08:39:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:39:51.845628 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596c43c3-db3e-473a-aeeb-45bf76ad7877 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:39:51.857294 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596c43c3-db3e-473a-aeeb-45bf76ad7877 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:39:51.860677 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fa7aa9-7e0e-4201-b9ed-0c36399876f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:39:51.861289 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fa7aa9-7e0e-4201-b9ed-0c36399876f7 service neutron] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:39:51.861569 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fa7aa9-7e0e-4201-b9ed-0c36399876f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:39:51.861836 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fa7aa9-7e0e-4201-b9ed-0c36399876f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:39:51.871421 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fa7aa9-7e0e-4201-b9ed-0c36399876f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:39:51.872840 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fa7aa9-7e0e-4201-b9ed-0c36399876f7 service neutron] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:39:51.882730 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 770/1533] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:39:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:39:55.435973 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21179461-038b-41a2-9b08-09a7b472abf4 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:39:55.436473 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21179461-038b-41a2-9b08-09a7b472abf4 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:39:55.436590 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21179461-038b-41a2-9b08-09a7b472abf4 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:39:55.437903 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 764/1534] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 08:39:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:39:55.443906 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afa8cfa-a22a-4628-8363-54fcabab253e None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:39:55.444349 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afa8cfa-a22a-4628-8363-54fcabab253e None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:39:55.444458 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afa8cfa-a22a-4628-8363-54fcabab253e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:39:55.452030 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=81 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:39:55.464337 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-2afa8cfa-a22a-4628-8363-54fcabab253e None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:39:55.474836 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-2afa8cfa-a22a-4628-8363-54fcabab253e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:39:55.474836 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 771/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 08:39:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:39:55.796115 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21179461-038b-41a2-9b08-09a7b472abf4 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:39:55.797825 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-21179461-038b-41a2-9b08-09a7b472abf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:39:55.861935 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:39:55.876221 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21179461-038b-41a2-9b08-09a7b472abf4 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:39:55.879869 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2aa0e6-22db-4a27-8147-198e2c1a3384 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ba74e28ced4cbba857f54222dfbe41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:39:55.880466 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2aa0e6-22db-4a27-8147-198e2c1a3384 service cinder] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:39:55.880745 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2aa0e6-22db-4a27-8147-198e2c1a3384 service cinder] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:39:55.881057 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2aa0e6-22db-4a27-8147-198e2c1a3384 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:39:55.890093 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2aa0e6-22db-4a27-8147-198e2c1a3384 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:39:55.891503 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2aa0e6-22db-4a27-8147-198e2c1a3384 service cinder] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:39:55.902178 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 765/1536] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:39:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:39:57.914885 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4586ec5d-9a63-403a-9cfd-c5d3eb3f0840 None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:39:57.915458 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4586ec5d-9a63-403a-9cfd-c5d3eb3f0840 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:39:57.915458 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4586ec5d-9a63-403a-9cfd-c5d3eb3f0840 None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:39:57.916808 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 772/1537] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 08:39:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:39:57.921987 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71587775-e22d-4d57-b346-de8c016ad16f None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:39:57.922396 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71587775-e22d-4d57-b346-de8c016ad16f None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:39:57.922528 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71587775-e22d-4d57-b346-de8c016ad16f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:39:57.942862 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-71587775-e22d-4d57-b346-de8c016ad16f None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:39:57.944134 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-71587775-e22d-4d57-b346-de8c016ad16f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:39:57.952028 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 766/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 08:39:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:39:58.255036 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4586ec5d-9a63-403a-9cfd-c5d3eb3f0840 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:39:58.256786 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-4586ec5d-9a63-403a-9cfd-c5d3eb3f0840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:39:58.271264 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4586ec5d-9a63-403a-9cfd-c5d3eb3f0840 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:39:58.274119 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ba1edc-9430-4922-b9fa-fa1e691d1e5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ba74e28ced4cbba857f54222dfbe41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:39:58.274716 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ba1edc-9430-4922-b9fa-fa1e691d1e5e service cinder] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:39:58.274993 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ba1edc-9430-4922-b9fa-fa1e691d1e5e service cinder] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:39:58.275264 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ba1edc-9430-4922-b9fa-fa1e691d1e5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:39:58.284093 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ba1edc-9430-4922-b9fa-fa1e691d1e5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:39:58.286232 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ba1edc-9430-4922-b9fa-fa1e691d1e5e service cinder] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:39:58.296347 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 773/1539] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:39:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:40:07.937513 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=11 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:40:08.292031 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=8 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:41:23.084943 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d988bde-8ad9-4e78-965e-ff4f0f2e61ec None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:23.086561 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d988bde-8ad9-4e78-965e-ff4f0f2e61ec None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:23.086561 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d988bde-8ad9-4e78-965e-ff4f0f2e61ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:23.111881 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-5d988bde-8ad9-4e78-965e-ff4f0f2e61ec None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:41:23.113144 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-5d988bde-8ad9-4e78-965e-ff4f0f2e61ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:41:23.120700 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9a94fd-6acd-47b4-9591-ec3584ae5c48 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:23.121340 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9a94fd-6acd-47b4-9591-ec3584ae5c48 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:23.121340 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9a94fd-6acd-47b4-9591-ec3584ae5c48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:23.128736 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 767/1540] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:41:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 08:41:23.148543 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-6c9a94fd-6acd-47b4-9591-ec3584ae5c48 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:41:23.149891 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-6c9a94fd-6acd-47b4-9591-ec3584ae5c48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:41:23.153923 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ef84af-326a-4865-8986-461896f8ec48 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:23.154206 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ef84af-326a-4865-8986-461896f8ec48 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:23.154511 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ef84af-326a-4865-8986-461896f8ec48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:23.162942 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 774/1541] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:41:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:41:23.171913 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4591a046-2d51-4585-8801-b204928bf06f None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:23.172270 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4591a046-2d51-4585-8801-b204928bf06f None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:23.172465 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4591a046-2d51-4585-8801-b204928bf06f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:23.181930 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-c9ef84af-326a-4865-8986-461896f8ec48 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:41:23.183141 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-c9ef84af-326a-4865-8986-461896f8ec48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:41:23.195060 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 768/1542] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:41:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 08:41:23.199439 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ef84af-326a-4865-8986-461896f8ec48 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:23.202697 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-4591a046-2d51-4585-8801-b204928bf06f None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:41:23.203913 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-4591a046-2d51-4585-8801-b204928bf06f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:41:23.215404 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ef84af-326a-4865-8986-461896f8ec48 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:23.221394 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 775/1543] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:41:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:41:23.222999 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b561918e-a9e0-4c0c-b6a5-db68f5b13e91 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:23.223444 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b561918e-a9e0-4c0c-b6a5-db68f5b13e91 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:23.223721 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b561918e-a9e0-4c0c-b6a5-db68f5b13e91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:23.233719 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7f3be0-2e9a-4330-8f57-8627ae51a4c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:23.234353 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7f3be0-2e9a-4330-8f57-8627ae51a4c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:23.234636 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7f3be0-2e9a-4330-8f57-8627ae51a4c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:23.234906 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7f3be0-2e9a-4330-8f57-8627ae51a4c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:23.235566 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7f3be0-2e9a-4330-8f57-8627ae51a4c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:41:23.236090 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7f3be0-2e9a-4330-8f57-8627ae51a4c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:23.238127 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7f3be0-2e9a-4330-8f57-8627ae51a4c1 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:23.251370 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-b561918e-a9e0-4c0c-b6a5-db68f5b13e91 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:41:23.252580 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-b561918e-a9e0-4c0c-b6a5-db68f5b13e91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:41:23.258962 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 769/1544] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Aug 10 08:41:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:41:23.259906 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7f3be0-2e9a-4330-8f57-8627ae51a4c1 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:23.268510 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 776/1545] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:41:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:41:23.269758 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b561918e-a9e0-4c0c-b6a5-db68f5b13e91 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:23.279237 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7f3be0-2e9a-4330-8f57-8627ae51a4c1 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:23.287747 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b561918e-a9e0-4c0c-b6a5-db68f5b13e91 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:23.293596 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5149bd6-1bdd-4eac-87e2-c52fd834e496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:23.294312 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5149bd6-1bdd-4eac-87e2-c52fd834e496 admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:23.294594 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5149bd6-1bdd-4eac-87e2-c52fd834e496 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:23.294866 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5149bd6-1bdd-4eac-87e2-c52fd834e496 admin admin] PATH_INFO: `/v3/projects` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:23.295491 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5149bd6-1bdd-4eac-87e2-c52fd834e496 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:23.297585 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91feec55-aabc-4d51-b0a9-3416e19867ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:23.297745 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5149bd6-1bdd-4eac-87e2-c52fd834e496 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:23.298206 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91feec55-aabc-4d51-b0a9-3416e19867ac admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:23.298547 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91feec55-aabc-4d51-b0a9-3416e19867ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:23.298951 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91feec55-aabc-4d51-b0a9-3416e19867ac admin admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:23.299608 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91feec55-aabc-4d51-b0a9-3416e19867ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:41:23.299914 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91feec55-aabc-4d51-b0a9-3416e19867ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:23.301664 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91feec55-aabc-4d51-b0a9-3416e19867ac admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:23.326735 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 770/1546] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Aug 10 08:41:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:41:23.328238 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9605f7ed-feb4-4bd8-aa8f-8b2222289661 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:23.328529 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9605f7ed-feb4-4bd8-aa8f-8b2222289661 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:23.329128 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9605f7ed-feb4-4bd8-aa8f-8b2222289661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:23.367051 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-9605f7ed-feb4-4bd8-aa8f-8b2222289661 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:41:23.367051 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-9605f7ed-feb4-4bd8-aa8f-8b2222289661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:41:23.369535 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 771/1547] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:41:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:41:23.370332 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9605f7ed-feb4-4bd8-aa8f-8b2222289661 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:23.383061 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9605f7ed-feb4-4bd8-aa8f-8b2222289661 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:23.387967 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb05240-a023-4d1f-a6ee-d857a0a4b239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:23.388573 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb05240-a023-4d1f-a6ee-d857a0a4b239 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:23.388859 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb05240-a023-4d1f-a6ee-d857a0a4b239 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:23.389199 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb05240-a023-4d1f-a6ee-d857a0a4b239 admin admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:23.389825 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb05240-a023-4d1f-a6ee-d857a0a4b239 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:23.390860 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: WARNING py.warnings [None req-adb05240-a023-4d1f-a6ee-d857a0a4b239 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 08:41:23.390860 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: warnings.warn(msg) Aug 10 08:41:23.390860 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: Aug 10 08:41:23.392037 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb05240-a023-4d1f-a6ee-d857a0a4b239 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:23.474829 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 777/1548] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 08:41:23 2022] POST /identity/v3/projects => generated 367 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:41:23.476243 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04030160-6b30-456d-9d2d-678d8775cdfd None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:23.476538 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04030160-6b30-456d-9d2d-678d8775cdfd None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:23.476809 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04030160-6b30-456d-9d2d-678d8775cdfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:23.495915 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-04030160-6b30-456d-9d2d-678d8775cdfd None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:41:23.499105 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-04030160-6b30-456d-9d2d-678d8775cdfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:41:23.505171 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 772/1549] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 08:41:23 2022] POST /identity/v3/projects => generated 383 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:41:23.506465 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2226da74-d97b-4a1d-a6ac-ebf43e1baded None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:23.506771 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2226da74-d97b-4a1d-a6ac-ebf43e1baded None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:23.507016 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2226da74-d97b-4a1d-a6ac-ebf43e1baded None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:23.527274 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-2226da74-d97b-4a1d-a6ac-ebf43e1baded None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:41:23.528455 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-2226da74-d97b-4a1d-a6ac-ebf43e1baded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:41:23.561878 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 778/1550] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:41:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:41:23.571568 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9e8917-e7e5-4f7c-9cbc-dd002d5a3465 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:23.571568 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9e8917-e7e5-4f7c-9cbc-dd002d5a3465 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:23.571568 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9e8917-e7e5-4f7c-9cbc-dd002d5a3465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:23.589646 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-cd9e8917-e7e5-4f7c-9cbc-dd002d5a3465 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:41:23.589853 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 773/1551] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:41:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:41:23.590933 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf7109f-70aa-4cbc-848b-7a8c582e36b4 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:23.591065 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-cd9e8917-e7e5-4f7c-9cbc-dd002d5a3465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:41:23.591227 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf7109f-70aa-4cbc-848b-7a8c582e36b4 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:23.591702 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf7109f-70aa-4cbc-848b-7a8c582e36b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:23.603854 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 779/1552] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:41:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:41:23.604876 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9e8917-e7e5-4f7c-9cbc-dd002d5a3465 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:23.611774 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-abf7109f-70aa-4cbc-848b-7a8c582e36b4 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:41:23.613515 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-abf7109f-70aa-4cbc-848b-7a8c582e36b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:41:23.617374 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9e8917-e7e5-4f7c-9cbc-dd002d5a3465 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:23.621731 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 774/1553] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:41:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:41:23.622905 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1a2583-e839-4dbb-9cec-40a677240ccc None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:23.623182 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1a2583-e839-4dbb-9cec-40a677240ccc None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:23.623450 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1a2583-e839-4dbb-9cec-40a677240ccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:23.625138 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296171e3-055d-4415-99a4-b893baa54b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:23.625792 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296171e3-055d-4415-99a4-b893baa54b1b admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:23.626075 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296171e3-055d-4415-99a4-b893baa54b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:23.626362 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296171e3-055d-4415-99a4-b893baa54b1b admin admin] PATH_INFO: `/v3/domains` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:23.627121 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296171e3-055d-4415-99a4-b893baa54b1b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:41:23.627533 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296171e3-055d-4415-99a4-b893baa54b1b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:23.629389 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: WARNING py.warnings [None req-296171e3-055d-4415-99a4-b893baa54b1b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 10 08:41:23.629389 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: warnings.warn(msg) Aug 10 08:41:23.629389 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: Aug 10 08:41:23.630382 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296171e3-055d-4415-99a4-b893baa54b1b admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:23.642777 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 780/1554] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Aug 10 08:41:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:41:23.643230 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296171e3-055d-4415-99a4-b893baa54b1b admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:23.650870 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-ba1a2583-e839-4dbb-9cec-40a677240ccc None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:41:23.652066 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-ba1a2583-e839-4dbb-9cec-40a677240ccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:41:23.657524 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296171e3-055d-4415-99a4-b893baa54b1b admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:23.660081 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 775/1555] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:41:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:41:23.661971 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1a2583-e839-4dbb-9cec-40a677240ccc None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:23.662813 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed99e753-5a2c-4c38-82ef-c188b4c8180c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:23.663414 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed99e753-5a2c-4c38-82ef-c188b4c8180c admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:23.663694 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed99e753-5a2c-4c38-82ef-c188b4c8180c admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:23.663965 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed99e753-5a2c-4c38-82ef-c188b4c8180c admin admin] PATH_INFO: `/v3/users` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:23.664580 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed99e753-5a2c-4c38-82ef-c188b4c8180c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:23.665629 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: WARNING py.warnings [None req-ed99e753-5a2c-4c38-82ef-c188b4c8180c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 08:41:23.665629 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: warnings.warn(msg) Aug 10 08:41:23.665629 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: Aug 10 08:41:23.666840 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed99e753-5a2c-4c38-82ef-c188b4c8180c admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:23.672418 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1a2583-e839-4dbb-9cec-40a677240ccc None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:23.678002 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5466eeea-bf16-4e2d-b1c6-f0d82be252fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:23.678594 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5466eeea-bf16-4e2d-b1c6-f0d82be252fa admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:23.678899 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5466eeea-bf16-4e2d-b1c6-f0d82be252fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:23.679164 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5466eeea-bf16-4e2d-b1c6-f0d82be252fa admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:23.679779 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5466eeea-bf16-4e2d-b1c6-f0d82be252fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:23.681724 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5466eeea-bf16-4e2d-b1c6-f0d82be252fa admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:23.759573 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 781/1556] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 08:41:23 2022] POST /identity/v3/users => generated 284 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:41:23.760976 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb61351-b744-425a-b509-924cccedef01 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:23.761261 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb61351-b744-425a-b509-924cccedef01 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:23.761530 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb61351-b744-425a-b509-924cccedef01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:23.771255 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 776/1557] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 08:41:23 2022] POST /identity/v3/users => generated 292 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:41:23.772578 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2990ea-cddb-46cf-9016-e885eb632fb0 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:23.772866 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2990ea-cddb-46cf-9016-e885eb632fb0 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:23.773174 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2990ea-cddb-46cf-9016-e885eb632fb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:23.781411 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-beb61351-b744-425a-b509-924cccedef01 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:41:23.782614 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-beb61351-b744-425a-b509-924cccedef01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:41:23.791701 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 782/1558] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:41:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:41:23.792321 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-ae2990ea-cddb-46cf-9016-e885eb632fb0 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:41:23.793103 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805f82ba-a581-48e7-93d4-c33d8496299d None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:23.793388 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805f82ba-a581-48e7-93d4-c33d8496299d None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:23.793517 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-ae2990ea-cddb-46cf-9016-e885eb632fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:41:23.793658 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805f82ba-a581-48e7-93d4-c33d8496299d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:23.805357 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 777/1559] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:41:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:41:23.805357 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a5eb45-577b-4134-b980-7b1913ce16ce None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:23.805357 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a5eb45-577b-4134-b980-7b1913ce16ce None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:23.805357 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a5eb45-577b-4134-b980-7b1913ce16ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:23.813112 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-805f82ba-a581-48e7-93d4-c33d8496299d None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:41:23.815016 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-805f82ba-a581-48e7-93d4-c33d8496299d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:41:23.822244 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-d1a5eb45-577b-4134-b980-7b1913ce16ce None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:41:23.826178 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 783/1560] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:41:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:41:23.826339 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-d1a5eb45-577b-4134-b980-7b1913ce16ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:41:23.826442 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805f82ba-a581-48e7-93d4-c33d8496299d None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:23.832055 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 778/1561] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:41:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:41:23.833516 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c64637-9d33-42d2-864e-4aa74adb3d57 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:23.833799 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c64637-9d33-42d2-864e-4aa74adb3d57 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:23.834072 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c64637-9d33-42d2-864e-4aa74adb3d57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:23.839284 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805f82ba-a581-48e7-93d4-c33d8496299d None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:23.844036 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78355392-9b1f-47cb-9e59-3d71ec0ed9fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:23.844634 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78355392-9b1f-47cb-9e59-3d71ec0ed9fb admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:23.844909 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78355392-9b1f-47cb-9e59-3d71ec0ed9fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:23.845199 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78355392-9b1f-47cb-9e59-3d71ec0ed9fb admin admin] PATH_INFO: `/v3/projects` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:23.845815 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78355392-9b1f-47cb-9e59-3d71ec0ed9fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:23.848347 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78355392-9b1f-47cb-9e59-3d71ec0ed9fb admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:23.853185 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-b9c64637-9d33-42d2-864e-4aa74adb3d57 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:41:23.854365 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-b9c64637-9d33-42d2-864e-4aa74adb3d57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:41:23.862146 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 779/1562] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:41:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:41:23.862952 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c64637-9d33-42d2-864e-4aa74adb3d57 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:23.872421 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c64637-9d33-42d2-864e-4aa74adb3d57 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:23.877115 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d6d3db-d06e-4113-9a56-5d437288b455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:23.877737 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d6d3db-d06e-4113-9a56-5d437288b455 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:23.878011 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d6d3db-d06e-4113-9a56-5d437288b455 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:23.878277 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d6d3db-d06e-4113-9a56-5d437288b455 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:23.878983 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d6d3db-d06e-4113-9a56-5d437288b455 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:23.880555 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d6d3db-d06e-4113-9a56-5d437288b455 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:23.890328 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 780/1563] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:41:23 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:41:23.891096 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d6d3db-d06e-4113-9a56-5d437288b455 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:23.901727 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d6d3db-d06e-4113-9a56-5d437288b455 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:23.906388 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d373fb5-48cf-4b47-83f9-bd82ce22fd3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:23.906954 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d373fb5-48cf-4b47-83f9-bd82ce22fd3a admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:23.907228 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d373fb5-48cf-4b47-83f9-bd82ce22fd3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:23.907495 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d373fb5-48cf-4b47-83f9-bd82ce22fd3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:23.908111 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d373fb5-48cf-4b47-83f9-bd82ce22fd3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:23.910890 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d373fb5-48cf-4b47-83f9-bd82ce22fd3a admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:23.915095 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 784/1564] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 08:41:23 2022] POST /identity/v3/projects => generated 387 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:41:23.915931 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78355392-9b1f-47cb-9e59-3d71ec0ed9fb admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:23.926592 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 781/1565] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:41:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:41:23.926592 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d373fb5-48cf-4b47-83f9-bd82ce22fd3a admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:23.934879 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d373fb5-48cf-4b47-83f9-bd82ce22fd3a admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:23.952966 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:41:23.962233 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:41:23.983208 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78355392-9b1f-47cb-9e59-3d71ec0ed9fb admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:23.988664 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f622160-654c-480c-8fc2-8e65d25dc4c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:23.989300 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f622160-654c-480c-8fc2-8e65d25dc4c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:23.989580 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f622160-654c-480c-8fc2-8e65d25dc4c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:23.989847 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f622160-654c-480c-8fc2-8e65d25dc4c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:23.990509 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f622160-654c-480c-8fc2-8e65d25dc4c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:41:23.990800 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f622160-654c-480c-8fc2-8e65d25dc4c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:23.992526 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f622160-654c-480c-8fc2-8e65d25dc4c1 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:23.993592 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fc00b4-7ee6-4bdc-bb92-8a3ed763974e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:23.994226 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fc00b4-7ee6-4bdc-bb92-8a3ed763974e admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:23.994507 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fc00b4-7ee6-4bdc-bb92-8a3ed763974e admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:23.994781 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fc00b4-7ee6-4bdc-bb92-8a3ed763974e admin admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:23.995527 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fc00b4-7ee6-4bdc-bb92-8a3ed763974e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:41:23.995839 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fc00b4-7ee6-4bdc-bb92-8a3ed763974e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:23.997466 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fc00b4-7ee6-4bdc-bb92-8a3ed763974e admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:24.005783 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 785/1566] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Aug 10 08:41:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:41:24.007356 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f622160-654c-480c-8fc2-8e65d25dc4c1 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:24.017587 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 782/1567] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Aug 10 08:41:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:41:24.017587 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fc00b4-7ee6-4bdc-bb92-8a3ed763974e admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:24.017877 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f622160-654c-480c-8fc2-8e65d25dc4c1 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:24.024058 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4f8a84-dd65-4225-b347-229df0626366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:24.024706 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4f8a84-dd65-4225-b347-229df0626366 admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:24.025017 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4f8a84-dd65-4225-b347-229df0626366 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:24.030146 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4f8a84-dd65-4225-b347-229df0626366 admin admin] PATH_INFO: `/v3/domains` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:24.030146 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4f8a84-dd65-4225-b347-229df0626366 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:41:24.030146 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4f8a84-dd65-4225-b347-229df0626366 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:24.030146 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4f8a84-dd65-4225-b347-229df0626366 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:24.030702 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fc00b4-7ee6-4bdc-bb92-8a3ed763974e admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:24.033322 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6f1cef-4c0f-4988-b828-f8e5d34117a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:24.033897 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6f1cef-4c0f-4988-b828-f8e5d34117a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:24.034202 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6f1cef-4c0f-4988-b828-f8e5d34117a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:24.034589 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6f1cef-4c0f-4988-b828-f8e5d34117a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:24.035817 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6f1cef-4c0f-4988-b828-f8e5d34117a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:41:24.036121 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6f1cef-4c0f-4988-b828-f8e5d34117a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:24.037724 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6f1cef-4c0f-4988-b828-f8e5d34117a1 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:24.040847 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 786/1568] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Aug 10 08:41:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:41:24.041698 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4f8a84-dd65-4225-b347-229df0626366 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:24.058015 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 783/1569] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Aug 10 08:41:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:41:24.058015 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6f1cef-4c0f-4988-b828-f8e5d34117a1 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:24.058317 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4f8a84-dd65-4225-b347-229df0626366 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:24.058317 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3e7536-0148-46f4-abc1-60fb5ba19309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:24.058317 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3e7536-0148-46f4-abc1-60fb5ba19309 admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:24.058317 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3e7536-0148-46f4-abc1-60fb5ba19309 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:24.058649 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3e7536-0148-46f4-abc1-60fb5ba19309 admin admin] PATH_INFO: `/v3/domains` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:24.058892 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3e7536-0148-46f4-abc1-60fb5ba19309 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:41:24.059163 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3e7536-0148-46f4-abc1-60fb5ba19309 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:24.060796 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3e7536-0148-46f4-abc1-60fb5ba19309 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:24.061614 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6f1cef-4c0f-4988-b828-f8e5d34117a1 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:24.067101 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e3f36a-ebda-4c9e-ad89-fc00ab41ada6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:24.067702 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e3f36a-ebda-4c9e-ad89-fc00ab41ada6 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:24.068413 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e3f36a-ebda-4c9e-ad89-fc00ab41ada6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:24.068413 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e3f36a-ebda-4c9e-ad89-fc00ab41ada6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:24.069023 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e3f36a-ebda-4c9e-ad89-fc00ab41ada6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:24.070601 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e3f36a-ebda-4c9e-ad89-fc00ab41ada6 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:24.077008 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 787/1570] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Aug 10 08:41:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:41:24.077807 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3e7536-0148-46f4-abc1-60fb5ba19309 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:24.084286 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 784/1571] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:41:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:41:24.084286 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e3f36a-ebda-4c9e-ad89-fc00ab41ada6 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:24.089844 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3e7536-0148-46f4-abc1-60fb5ba19309 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:24.094328 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a2e521-e008-4b55-9d6b-4ae25dd9cda2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:24.094812 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e3f36a-ebda-4c9e-ad89-fc00ab41ada6 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:24.095345 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a2e521-e008-4b55-9d6b-4ae25dd9cda2 admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:24.095345 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a2e521-e008-4b55-9d6b-4ae25dd9cda2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:24.095569 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a2e521-e008-4b55-9d6b-4ae25dd9cda2 admin admin] PATH_INFO: `/v3/users` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:24.096100 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a2e521-e008-4b55-9d6b-4ae25dd9cda2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:24.098682 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a2e521-e008-4b55-9d6b-4ae25dd9cda2 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:24.100451 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fe0e03-ea3d-4814-884f-446b4f1d47e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:24.101063 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fe0e03-ea3d-4814-884f-446b4f1d47e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:24.101349 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fe0e03-ea3d-4814-884f-446b4f1d47e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:24.101621 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fe0e03-ea3d-4814-884f-446b4f1d47e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:24.102254 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fe0e03-ea3d-4814-884f-446b4f1d47e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:24.103831 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fe0e03-ea3d-4814-884f-446b4f1d47e1 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:24.114819 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 785/1572] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:41:24 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:41:24.117750 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fe0e03-ea3d-4814-884f-446b4f1d47e1 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:24.125264 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fe0e03-ea3d-4814-884f-446b4f1d47e1 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:24.129772 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28503bd5-99cb-4ded-bfb2-fb1c69698804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:24.130329 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28503bd5-99cb-4ded-bfb2-fb1c69698804 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:24.130607 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28503bd5-99cb-4ded-bfb2-fb1c69698804 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:24.130879 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28503bd5-99cb-4ded-bfb2-fb1c69698804 admin admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:24.131482 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28503bd5-99cb-4ded-bfb2-fb1c69698804 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:24.133389 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28503bd5-99cb-4ded-bfb2-fb1c69698804 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:24.180121 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 788/1573] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 08:41:24 2022] POST /identity/v3/users => generated 294 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:41:24.180121 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a2e521-e008-4b55-9d6b-4ae25dd9cda2 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:24.187333 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a2e521-e008-4b55-9d6b-4ae25dd9cda2 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:24.192371 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b390a9-04d6-497c-a653-b41a06859fe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:24.192969 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b390a9-04d6-497c-a653-b41a06859fe4 admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:24.193249 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b390a9-04d6-497c-a653-b41a06859fe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:24.193519 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b390a9-04d6-497c-a653-b41a06859fe4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:24.194121 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b390a9-04d6-497c-a653-b41a06859fe4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:24.196088 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b390a9-04d6-497c-a653-b41a06859fe4 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:24.223586 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 786/1574] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 08:41:24 2022] POST /identity/v3/projects => generated 391 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:41:24.224727 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28503bd5-99cb-4ded-bfb2-fb1c69698804 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:24.284020 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 789/1575] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 08:41:24 2022] POST /identity/v3/projects => generated 385 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:41:24.284020 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b390a9-04d6-497c-a653-b41a06859fe4 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:24.285864 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28503bd5-99cb-4ded-bfb2-fb1c69698804 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:24.344842 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f386a364-ce36-4c20-8770-f8333f9ad030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:24.345089 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b390a9-04d6-497c-a653-b41a06859fe4 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:24.345500 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f386a364-ce36-4c20-8770-f8333f9ad030 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:24.345800 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f386a364-ce36-4c20-8770-f8333f9ad030 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:24.346065 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f386a364-ce36-4c20-8770-f8333f9ad030 admin admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:24.346678 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f386a364-ce36-4c20-8770-f8333f9ad030 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:24.349121 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f386a364-ce36-4c20-8770-f8333f9ad030 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:24.350596 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c73b7e-6ba5-46b3-8db6-acacdae13feb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:24.351268 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c73b7e-6ba5-46b3-8db6-acacdae13feb admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:24.351687 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c73b7e-6ba5-46b3-8db6-acacdae13feb admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:24.352038 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c73b7e-6ba5-46b3-8db6-acacdae13feb admin admin] PATH_INFO: `/v3/projects` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:24.352702 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c73b7e-6ba5-46b3-8db6-acacdae13feb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:24.354755 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c73b7e-6ba5-46b3-8db6-acacdae13feb admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:24.358592 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:41:24.417971 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 787/1576] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 08:41:24 2022] POST /identity/v3/projects => generated 377 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:41:24.418931 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f386a364-ce36-4c20-8770-f8333f9ad030 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:24.434414 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 790/1577] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 08:41:24 2022] POST /identity/v3/projects => generated 373 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:41:24.437915 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c73b7e-6ba5-46b3-8db6-acacdae13feb admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:24.483237 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f386a364-ce36-4c20-8770-f8333f9ad030 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:24.488744 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:41:24.500128 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c73b7e-6ba5-46b3-8db6-acacdae13feb admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:24.505210 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12080f30-8563-4bbe-a04e-0f8611b03666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:24.505819 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12080f30-8563-4bbe-a04e-0f8611b03666 admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:24.506088 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12080f30-8563-4bbe-a04e-0f8611b03666 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:24.506351 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12080f30-8563-4bbe-a04e-0f8611b03666 admin admin] PATH_INFO: `/v3/projects` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:24.506965 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12080f30-8563-4bbe-a04e-0f8611b03666 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:24.508892 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12080f30-8563-4bbe-a04e-0f8611b03666 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:24.540498 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816de85d-a005-4117-8764-98d0534bf509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:24.541173 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816de85d-a005-4117-8764-98d0534bf509 admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:24.541455 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816de85d-a005-4117-8764-98d0534bf509 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:24.541729 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816de85d-a005-4117-8764-98d0534bf509 admin admin] PATH_INFO: `/v3/projects/6f9f6bd4786942b8a5297dd04d0c79ba/users/e43802beaa824a5cb00759664f5bd646/roles/44b64db484f24a46bf0b7ced46b519bf` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:24.556277 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816de85d-a005-4117-8764-98d0534bf509 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f9f6bd4786942b8a5297dd04d0c79ba, user_id=e43802beaa824a5cb00759664f5bd646, role_id=44b64db484f24a46bf0b7ced46b519bf)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:24.562432 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: WARNING py.warnings [None req-816de85d-a005-4117-8764-98d0534bf509 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 08:41:24.562432 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: warnings.warn(msg) Aug 10 08:41:24.562432 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: Aug 10 08:41:24.562432 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816de85d-a005-4117-8764-98d0534bf509 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:24.588569 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 791/1578] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 08:41:24 2022] POST /identity/v3/projects => generated 379 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:41:24.589409 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12080f30-8563-4bbe-a04e-0f8611b03666 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:24.649007 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12080f30-8563-4bbe-a04e-0f8611b03666 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:24.653629 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6d53bc-d8a6-4633-af71-aa8a3bb3bc96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:24.654217 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6d53bc-d8a6-4633-af71-aa8a3bb3bc96 admin admin] REQUEST_METHOD: `PUT` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:24.654489 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6d53bc-d8a6-4633-af71-aa8a3bb3bc96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:24.654758 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6d53bc-d8a6-4633-af71-aa8a3bb3bc96 admin admin] PATH_INFO: `/v3/projects/e7ab7cc7a6364f5bad3ca36b070968f8/users/258ec0f975eb47738f4dac356c0578e6/roles/44b64db484f24a46bf0b7ced46b519bf` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:24.667548 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 788/1579] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 08:41:24 2022] PUT /identity/v3/projects/6f9f6bd4786942b8a5297dd04d0c79ba/users/e43802beaa824a5cb00759664f5bd646/roles/44b64db484f24a46bf0b7ced46b519bf => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:41:24.668456 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816de85d-a005-4117-8764-98d0534bf509 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:24.674148 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6d53bc-d8a6-4633-af71-aa8a3bb3bc96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7ab7cc7a6364f5bad3ca36b070968f8, user_id=258ec0f975eb47738f4dac356c0578e6, role_id=44b64db484f24a46bf0b7ced46b519bf)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:24.677072 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6d53bc-d8a6-4633-af71-aa8a3bb3bc96 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:24.733231 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816de85d-a005-4117-8764-98d0534bf509 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:24.738152 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2c562f-c646-4889-8577-81ef0494b6c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:24.738704 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2c562f-c646-4889-8577-81ef0494b6c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:24.738983 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2c562f-c646-4889-8577-81ef0494b6c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:24.739252 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2c562f-c646-4889-8577-81ef0494b6c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:24.739893 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2c562f-c646-4889-8577-81ef0494b6c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:24.741519 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2c562f-c646-4889-8577-81ef0494b6c8 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:24.751646 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 789/1580] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:41:24 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:41:24.752566 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2c562f-c646-4889-8577-81ef0494b6c8 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:24.762470 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2c562f-c646-4889-8577-81ef0494b6c8 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:24.767090 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cdbccc-b7ca-4fd3-aa0f-5b9dd753fa3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:24.767639 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cdbccc-b7ca-4fd3-aa0f-5b9dd753fa3c admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:24.767929 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cdbccc-b7ca-4fd3-aa0f-5b9dd753fa3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:24.768189 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cdbccc-b7ca-4fd3-aa0f-5b9dd753fa3c admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:24.769380 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cdbccc-b7ca-4fd3-aa0f-5b9dd753fa3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:24.771332 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cdbccc-b7ca-4fd3-aa0f-5b9dd753fa3c admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:24.776296 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 792/1581] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 08:41:24 2022] PUT /identity/v3/projects/e7ab7cc7a6364f5bad3ca36b070968f8/users/258ec0f975eb47738f4dac356c0578e6/roles/44b64db484f24a46bf0b7ced46b519bf => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:41:24.780535 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6d53bc-d8a6-4633-af71-aa8a3bb3bc96 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:24.840667 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6d53bc-d8a6-4633-af71-aa8a3bb3bc96 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:24.845294 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196d7f22-8847-4503-a7ba-5773045bfe0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:24.847559 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196d7f22-8847-4503-a7ba-5773045bfe0c admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:24.847559 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196d7f22-8847-4503-a7ba-5773045bfe0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:24.847559 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196d7f22-8847-4503-a7ba-5773045bfe0c admin admin] PATH_INFO: `/v3/users` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:24.847559 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196d7f22-8847-4503-a7ba-5773045bfe0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:24.849107 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196d7f22-8847-4503-a7ba-5773045bfe0c admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:24.867248 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:41:24.873238 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 790/1582] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 08:41:24 2022] POST /identity/v3/users => generated 296 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:41:24.875028 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cdbccc-b7ca-4fd3-aa0f-5b9dd753fa3c admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:24.885224 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cdbccc-b7ca-4fd3-aa0f-5b9dd753fa3c admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:24.889823 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c34314-cd48-4f72-95c0-837760bc3dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:24.890398 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c34314-cd48-4f72-95c0-837760bc3dfe admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:24.890676 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c34314-cd48-4f72-95c0-837760bc3dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:24.892266 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c34314-cd48-4f72-95c0-837760bc3dfe admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:24.892266 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c34314-cd48-4f72-95c0-837760bc3dfe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:24.893525 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c34314-cd48-4f72-95c0-837760bc3dfe admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:24.919047 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 793/1583] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 08:41:24 2022] POST /identity/v3/users => generated 293 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:41:24.919921 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196d7f22-8847-4503-a7ba-5773045bfe0c admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:24.929273 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196d7f22-8847-4503-a7ba-5773045bfe0c admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:24.933779 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f4b65b-e6fa-4658-967e-c00f0f3f3e72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:24.934345 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f4b65b-e6fa-4658-967e-c00f0f3f3e72 admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:24.934620 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f4b65b-e6fa-4658-967e-c00f0f3f3e72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:24.934914 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f4b65b-e6fa-4658-967e-c00f0f3f3e72 admin admin] PATH_INFO: `/v3/users` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:24.935482 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f4b65b-e6fa-4658-967e-c00f0f3f3e72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:24.937420 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f4b65b-e6fa-4658-967e-c00f0f3f3e72 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:24.987781 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 791/1584] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 08:41:24 2022] POST /identity/v3/users => generated 289 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:41:24.988592 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c34314-cd48-4f72-95c0-837760bc3dfe admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:24.998164 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c34314-cd48-4f72-95c0-837760bc3dfe admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:25.002976 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e67bc9-2631-4654-a6d0-874dd9e35ef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:25.004081 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e67bc9-2631-4654-a6d0-874dd9e35ef2 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:25.004354 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e67bc9-2631-4654-a6d0-874dd9e35ef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:25.004617 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e67bc9-2631-4654-a6d0-874dd9e35ef2 admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:25.005259 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e67bc9-2631-4654-a6d0-874dd9e35ef2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:25.007137 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e67bc9-2631-4654-a6d0-874dd9e35ef2 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:25.016143 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 794/1585] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 08:41:24 2022] POST /identity/v3/users => generated 287 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:41:25.017740 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ffec58-b1a6-4691-b450-57a2419bf25f None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:25.018027 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ffec58-b1a6-4691-b450-57a2419bf25f None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:25.018304 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ffec58-b1a6-4691-b450-57a2419bf25f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:25.045782 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-90ffec58-b1a6-4691-b450-57a2419bf25f None None] MFA Rules not processed for user `e43802beaa824a5cb00759664f5bd646`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:41:25.046985 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-90ffec58-b1a6-4691-b450-57a2419bf25f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:41:25.064642 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:41:25.096706 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 792/1586] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 08:41:24 2022] POST /identity/v3/users => generated 290 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:41:25.097913 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e67bc9-2631-4654-a6d0-874dd9e35ef2 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:25.107576 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e67bc9-2631-4654-a6d0-874dd9e35ef2 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:25.111575 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 795/1587] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:41:25 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:41:25.112369 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c78270-881c-46d1-aff6-1663f6cadd5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:25.113516 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c78270-881c-46d1-aff6-1663f6cadd5b admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:25.113802 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c78270-881c-46d1-aff6-1663f6cadd5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:25.113954 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24dfc57-bd38-47dc-8e5b-121331d4a81d None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:25.114072 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c78270-881c-46d1-aff6-1663f6cadd5b admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:25.114172 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24dfc57-bd38-47dc-8e5b-121331d4a81d None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:25.114403 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24dfc57-bd38-47dc-8e5b-121331d4a81d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:25.114843 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c78270-881c-46d1-aff6-1663f6cadd5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:25.116577 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c78270-881c-46d1-aff6-1663f6cadd5b admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:25.131313 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 793/1588] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:41:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:41:25.131313 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c78270-881c-46d1-aff6-1663f6cadd5b admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:25.135438 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-a24dfc57-bd38-47dc-8e5b-121331d4a81d None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:41:25.136622 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-a24dfc57-bd38-47dc-8e5b-121331d4a81d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:41:25.141130 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c78270-881c-46d1-aff6-1663f6cadd5b admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:25.144760 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 796/1589] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:41:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:41:25.145604 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24dfc57-bd38-47dc-8e5b-121331d4a81d None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:25.149592 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0420a5a-4ba2-46bc-ac94-6b322f982519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:25.150643 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0420a5a-4ba2-46bc-ac94-6b322f982519 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:25.151500 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0420a5a-4ba2-46bc-ac94-6b322f982519 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:25.151500 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0420a5a-4ba2-46bc-ac94-6b322f982519 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:25.151902 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0420a5a-4ba2-46bc-ac94-6b322f982519 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:25.153694 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0420a5a-4ba2-46bc-ac94-6b322f982519 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:25.155040 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24dfc57-bd38-47dc-8e5b-121331d4a81d None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:25.160795 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2d4913-4c4f-400d-8934-2abc84e5fcac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:25.161417 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2d4913-4c4f-400d-8934-2abc84e5fcac admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:25.161695 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2d4913-4c4f-400d-8934-2abc84e5fcac admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:25.161961 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2d4913-4c4f-400d-8934-2abc84e5fcac admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:25.162593 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2d4913-4c4f-400d-8934-2abc84e5fcac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:25.164179 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2d4913-4c4f-400d-8934-2abc84e5fcac admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:25.167687 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 794/1590] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:41:25 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:41:25.167687 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0420a5a-4ba2-46bc-ac94-6b322f982519 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:25.176058 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0420a5a-4ba2-46bc-ac94-6b322f982519 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:25.177760 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 797/1591] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:41:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:41:25.178554 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2d4913-4c4f-400d-8934-2abc84e5fcac admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:25.183749 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1051d3d3-0607-40b7-a9b5-eb98ad33badf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:25.184351 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1051d3d3-0607-40b7-a9b5-eb98ad33badf admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:25.184632 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1051d3d3-0607-40b7-a9b5-eb98ad33badf admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:25.184899 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1051d3d3-0607-40b7-a9b5-eb98ad33badf admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:25.185564 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1051d3d3-0607-40b7-a9b5-eb98ad33badf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:25.187151 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1051d3d3-0607-40b7-a9b5-eb98ad33badf admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:25.187823 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2d4913-4c4f-400d-8934-2abc84e5fcac admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:25.193682 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad42ddf0-8bef-4d57-a00c-e581039aa4be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:25.194264 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad42ddf0-8bef-4d57-a00c-e581039aa4be admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:25.194653 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad42ddf0-8bef-4d57-a00c-e581039aa4be admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:25.194928 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad42ddf0-8bef-4d57-a00c-e581039aa4be admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:25.195652 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad42ddf0-8bef-4d57-a00c-e581039aa4be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:25.197278 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad42ddf0-8bef-4d57-a00c-e581039aa4be admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:25.201902 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 795/1592] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:41:25 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:41:25.201902 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1051d3d3-0607-40b7-a9b5-eb98ad33badf admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:25.210369 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 798/1593] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:41:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:41:25.210369 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad42ddf0-8bef-4d57-a00c-e581039aa4be admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:25.213062 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1051d3d3-0607-40b7-a9b5-eb98ad33badf admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:25.217650 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b9b638-0459-4de8-afdc-05e3383a601a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:25.218239 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b9b638-0459-4de8-afdc-05e3383a601a admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:25.218551 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b9b638-0459-4de8-afdc-05e3383a601a admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:25.218823 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b9b638-0459-4de8-afdc-05e3383a601a admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:25.219458 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b9b638-0459-4de8-afdc-05e3383a601a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:25.221084 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b9b638-0459-4de8-afdc-05e3383a601a admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:25.231047 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 796/1594] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:41:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:41:25.231819 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b9b638-0459-4de8-afdc-05e3383a601a admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:25.241590 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b9b638-0459-4de8-afdc-05e3383a601a admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:25.246325 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a936cc-418b-49a2-83cc-d744fc1926ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:25.247022 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a936cc-418b-49a2-83cc-d744fc1926ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:25.247303 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a936cc-418b-49a2-83cc-d744fc1926ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:25.247574 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a936cc-418b-49a2-83cc-d744fc1926ba admin admin] PATH_INFO: `/v3/projects/37950ef781584676ae470fd2c6394ae3/users/3bf116fade674838b9219e15dd20eda2/roles/44b64db484f24a46bf0b7ced46b519bf` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:25.263366 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a936cc-418b-49a2-83cc-d744fc1926ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=37950ef781584676ae470fd2c6394ae3, user_id=3bf116fade674838b9219e15dd20eda2, role_id=44b64db484f24a46bf0b7ced46b519bf)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:25.264715 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad42ddf0-8bef-4d57-a00c-e581039aa4be admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:25.266047 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a936cc-418b-49a2-83cc-d744fc1926ba admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:25.269432 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46712d1a-1a07-4dbb-9bb1-3ca06880dd9e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ba74e28ced4cbba857f54222dfbe41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:25.270046 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46712d1a-1a07-4dbb-9bb1-3ca06880dd9e service cinder] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:25.270329 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46712d1a-1a07-4dbb-9bb1-3ca06880dd9e service cinder] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:25.270632 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46712d1a-1a07-4dbb-9bb1-3ca06880dd9e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:25.279244 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46712d1a-1a07-4dbb-9bb1-3ca06880dd9e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:25.285672 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46712d1a-1a07-4dbb-9bb1-3ca06880dd9e service cinder] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:25.290085 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 799/1595] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:41:25 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:41:25.290857 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46712d1a-1a07-4dbb-9bb1-3ca06880dd9e service cinder] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:25.346049 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46712d1a-1a07-4dbb-9bb1-3ca06880dd9e service cinder] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:25.350098 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6461912e-e80f-4289-90b4-5945e27c0aab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:25.350814 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6461912e-e80f-4289-90b4-5945e27c0aab service neutron] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:25.351105 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6461912e-e80f-4289-90b4-5945e27c0aab service neutron] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:25.351373 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6461912e-e80f-4289-90b4-5945e27c0aab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:25.361192 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6461912e-e80f-4289-90b4-5945e27c0aab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:25.363432 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6461912e-e80f-4289-90b4-5945e27c0aab service neutron] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:25.365843 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 797/1596] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 08:41:25 2022] PUT /identity/v3/projects/37950ef781584676ae470fd2c6394ae3/users/3bf116fade674838b9219e15dd20eda2/roles/44b64db484f24a46bf0b7ced46b519bf => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:41:25.370127 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a936cc-418b-49a2-83cc-d744fc1926ba admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:25.374394 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 800/1597] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:41:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:41:25.375285 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6461912e-e80f-4289-90b4-5945e27c0aab service neutron] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:25.392433 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6461912e-e80f-4289-90b4-5945e27c0aab service neutron] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:25.443323 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:41:25.463121 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a936cc-418b-49a2-83cc-d744fc1926ba admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:25.468204 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6608d053-73fc-41a9-a9e7-20b429b3a260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:25.468350 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8e964f-e3bc-4bb5-b503-6b55e19c7ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:25.468720 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6608d053-73fc-41a9-a9e7-20b429b3a260 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:25.468843 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8e964f-e3bc-4bb5-b503-6b55e19c7ffe admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:25.469024 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6608d053-73fc-41a9-a9e7-20b429b3a260 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:25.469154 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8e964f-e3bc-4bb5-b503-6b55e19c7ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:25.469434 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6608d053-73fc-41a9-a9e7-20b429b3a260 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:25.469567 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8e964f-e3bc-4bb5-b503-6b55e19c7ffe admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:25.470139 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6608d053-73fc-41a9-a9e7-20b429b3a260 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:25.470310 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8e964f-e3bc-4bb5-b503-6b55e19c7ffe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:25.472386 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6608d053-73fc-41a9-a9e7-20b429b3a260 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:25.472521 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8e964f-e3bc-4bb5-b503-6b55e19c7ffe admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:25.488334 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 798/1598] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:41:25 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:41:25.488334 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6608d053-73fc-41a9-a9e7-20b429b3a260 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:25.492061 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 801/1599] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:41:25 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:41:25.493287 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8e964f-e3bc-4bb5-b503-6b55e19c7ffe admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:25.500475 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6608d053-73fc-41a9-a9e7-20b429b3a260 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:25.506874 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8e964f-e3bc-4bb5-b503-6b55e19c7ffe admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:25.508446 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97aaf4c-444b-4ca9-8c5a-5d5e729094d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:25.509088 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97aaf4c-444b-4ca9-8c5a-5d5e729094d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:25.509361 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97aaf4c-444b-4ca9-8c5a-5d5e729094d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:25.509627 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97aaf4c-444b-4ca9-8c5a-5d5e729094d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:25.510413 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97aaf4c-444b-4ca9-8c5a-5d5e729094d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:25.511546 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0427c75-a3b8-476d-be0a-170e88f77941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:25.512008 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97aaf4c-444b-4ca9-8c5a-5d5e729094d9 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:25.512342 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0427c75-a3b8-476d-be0a-170e88f77941 admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:25.512725 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0427c75-a3b8-476d-be0a-170e88f77941 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:25.513293 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0427c75-a3b8-476d-be0a-170e88f77941 admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:25.513955 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0427c75-a3b8-476d-be0a-170e88f77941 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:25.515650 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0427c75-a3b8-476d-be0a-170e88f77941 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:25.524008 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 799/1600] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:41:25 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:41:25.524747 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97aaf4c-444b-4ca9-8c5a-5d5e729094d9 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:25.532742 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 802/1601] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:41:25 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:41:25.534854 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8935da37-e574-4c8f-8931-9298f0d75d17 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:25.535434 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8935da37-e574-4c8f-8931-9298f0d75d17 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:25.535937 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8935da37-e574-4c8f-8931-9298f0d75d17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:25.536146 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97aaf4c-444b-4ca9-8c5a-5d5e729094d9 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:25.541627 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e92646a-9892-4a41-b205-f24f033441cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:25.542237 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e92646a-9892-4a41-b205-f24f033441cd admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:25.542517 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e92646a-9892-4a41-b205-f24f033441cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:25.542789 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e92646a-9892-4a41-b205-f24f033441cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:25.543573 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e92646a-9892-4a41-b205-f24f033441cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:25.545210 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e92646a-9892-4a41-b205-f24f033441cd admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:25.557771 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 800/1602] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:41:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:41:25.558519 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e92646a-9892-4a41-b205-f24f033441cd admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:25.562584 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-8935da37-e574-4c8f-8931-9298f0d75d17 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:41:25.562584 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-8935da37-e574-4c8f-8931-9298f0d75d17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:41:25.571208 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e92646a-9892-4a41-b205-f24f033441cd admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:25.572395 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 803/1603] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:41:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:41:25.573670 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8935da37-e574-4c8f-8931-9298f0d75d17 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:25.579705 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc3ff82-181b-48d1-961b-c10b4c829009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:25.580467 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc3ff82-181b-48d1-961b-c10b4c829009 admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:25.580737 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc3ff82-181b-48d1-961b-c10b4c829009 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:25.581101 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc3ff82-181b-48d1-961b-c10b4c829009 admin admin] PATH_INFO: `/v3/projects/aa5abc2b996a4056b005a992b5d0a7e9/users/4ec7ba61e65447a5a87b0f8fcc2b946c/roles/44b64db484f24a46bf0b7ced46b519bf` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:25.586573 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8935da37-e574-4c8f-8931-9298f0d75d17 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:25.592031 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26348434-6572-4a79-9428-b22720c0462b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:25.593010 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26348434-6572-4a79-9428-b22720c0462b admin admin] REQUEST_METHOD: `PUT` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:25.593523 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26348434-6572-4a79-9428-b22720c0462b admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:25.594145 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26348434-6572-4a79-9428-b22720c0462b admin admin] PATH_INFO: `/v3/projects/72624a2133804ace9905a30f82bf1148/users/8622a62d2fd4484c9444bc4348cecdfe/roles/44b64db484f24a46bf0b7ced46b519bf` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:25.600787 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc3ff82-181b-48d1-961b-c10b4c829009 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa5abc2b996a4056b005a992b5d0a7e9, user_id=4ec7ba61e65447a5a87b0f8fcc2b946c, role_id=44b64db484f24a46bf0b7ced46b519bf)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:25.603692 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc3ff82-181b-48d1-961b-c10b4c829009 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:25.611955 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26348434-6572-4a79-9428-b22720c0462b admin admin] RBAC: Authorizing `identity:create_grant(project_id=72624a2133804ace9905a30f82bf1148, user_id=8622a62d2fd4484c9444bc4348cecdfe, role_id=44b64db484f24a46bf0b7ced46b519bf)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:25.615234 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26348434-6572-4a79-9428-b22720c0462b admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:25.702774 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 801/1604] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 08:41:25 2022] PUT /identity/v3/projects/aa5abc2b996a4056b005a992b5d0a7e9/users/4ec7ba61e65447a5a87b0f8fcc2b946c/roles/44b64db484f24a46bf0b7ced46b519bf => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:41:25.703230 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 804/1605] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 08:41:25 2022] PUT /identity/v3/projects/72624a2133804ace9905a30f82bf1148/users/8622a62d2fd4484c9444bc4348cecdfe/roles/44b64db484f24a46bf0b7ced46b519bf => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:41:25.704054 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26348434-6572-4a79-9428-b22720c0462b admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:25.704166 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc3ff82-181b-48d1-961b-c10b4c829009 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:25.718904 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:41:25.796787 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26348434-6572-4a79-9428-b22720c0462b admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:25.804384 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fd45b5-e9ba-4bf3-b9fc-13cb76fd05c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:25.809927 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fd45b5-e9ba-4bf3-b9fc-13cb76fd05c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:25.809927 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fd45b5-e9ba-4bf3-b9fc-13cb76fd05c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:25.809927 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fd45b5-e9ba-4bf3-b9fc-13cb76fd05c9 admin admin] PATH_INFO: `/v3/projects/dd8d68233ef74767a19a50a277a495ad/users/ff34a3644dc84bb49cca4de7f59c1d31/roles/44b64db484f24a46bf0b7ced46b519bf` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:25.810254 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc3ff82-181b-48d1-961b-c10b4c829009 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:25.816522 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4893ab-f961-49eb-9dcc-504fe3009d26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:25.817402 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4893ab-f961-49eb-9dcc-504fe3009d26 admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:25.817915 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4893ab-f961-49eb-9dcc-504fe3009d26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:25.818288 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4893ab-f961-49eb-9dcc-504fe3009d26 admin admin] PATH_INFO: `/v3/projects/95db0a5b805944418265a85dc6cef4fc/users/9797df3e05944552b741f53773abf80a/roles/44b64db484f24a46bf0b7ced46b519bf` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:25.827198 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fd45b5-e9ba-4bf3-b9fc-13cb76fd05c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd8d68233ef74767a19a50a277a495ad, user_id=ff34a3644dc84bb49cca4de7f59c1d31, role_id=44b64db484f24a46bf0b7ced46b519bf)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:25.830083 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fd45b5-e9ba-4bf3-b9fc-13cb76fd05c9 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:25.836753 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4893ab-f961-49eb-9dcc-504fe3009d26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95db0a5b805944418265a85dc6cef4fc, user_id=9797df3e05944552b741f53773abf80a, role_id=44b64db484f24a46bf0b7ced46b519bf)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:25.839921 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4893ab-f961-49eb-9dcc-504fe3009d26 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:25.902924 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 802/1606] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 08:41:25 2022] PUT /identity/v3/projects/95db0a5b805944418265a85dc6cef4fc/users/9797df3e05944552b741f53773abf80a/roles/44b64db484f24a46bf0b7ced46b519bf => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:41:25.903489 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 805/1607] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 08:41:25 2022] PUT /identity/v3/projects/dd8d68233ef74767a19a50a277a495ad/users/ff34a3644dc84bb49cca4de7f59c1d31/roles/44b64db484f24a46bf0b7ced46b519bf => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:41:25.906916 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fd45b5-e9ba-4bf3-b9fc-13cb76fd05c9 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:25.906916 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-22fd45b5-e9ba-4bf3-b9fc-13cb76fd05c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:41:25.907550 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4893ab-f961-49eb-9dcc-504fe3009d26 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:25.980338 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4893ab-f961-49eb-9dcc-504fe3009d26 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:25.993973 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fd45b5-e9ba-4bf3-b9fc-13cb76fd05c9 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:25.998415 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4887ad3d-56a1-458b-b688-0b0da0ad1d53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:26.004583 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4887ad3d-56a1-458b-b688-0b0da0ad1d53 service neutron] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:26.004583 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4887ad3d-56a1-458b-b688-0b0da0ad1d53 service neutron] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:26.004583 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4887ad3d-56a1-458b-b688-0b0da0ad1d53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:26.043040 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:41:26.066926 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5c5050-409c-40e6-8b0f-d824a2565de8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:26.067594 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5c5050-409c-40e6-8b0f-d824a2565de8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:26.067872 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5c5050-409c-40e6-8b0f-d824a2565de8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:26.068147 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5c5050-409c-40e6-8b0f-d824a2565de8 admin admin] PATH_INFO: `/v3/projects/1cdf045fd98443afa186f8d3e2bf690e/users/b82a623527ad4280a353270fc3d51ba1/roles/44b64db484f24a46bf0b7ced46b519bf` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:26.080728 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4887ad3d-56a1-458b-b688-0b0da0ad1d53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:26.083093 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4887ad3d-56a1-458b-b688-0b0da0ad1d53 service neutron] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:26.083546 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5c5050-409c-40e6-8b0f-d824a2565de8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cdf045fd98443afa186f8d3e2bf690e, user_id=b82a623527ad4280a353270fc3d51ba1, role_id=44b64db484f24a46bf0b7ced46b519bf)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:26.086441 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5c5050-409c-40e6-8b0f-d824a2565de8 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:26.092582 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 806/1608] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:41:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:41:26.093930 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ed1e6d-2a2a-4bd8-bbbd-dfc107778084 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:26.094205 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ed1e6d-2a2a-4bd8-bbbd-dfc107778084 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:26.094469 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ed1e6d-2a2a-4bd8-bbbd-dfc107778084 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:26.115506 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-81ed1e6d-2a2a-4bd8-bbbd-dfc107778084 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:41:26.116683 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-81ed1e6d-2a2a-4bd8-bbbd-dfc107778084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:41:26.125484 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 807/1609] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:41:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:41:26.126765 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad13dc9e-cc90-4651-b8d5-4532cbb5f605 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:26.127050 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad13dc9e-cc90-4651-b8d5-4532cbb5f605 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:26.127317 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad13dc9e-cc90-4651-b8d5-4532cbb5f605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:26.158808 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-ad13dc9e-cc90-4651-b8d5-4532cbb5f605 None None] MFA Rules not processed for user `8622a62d2fd4484c9444bc4348cecdfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:41:26.165136 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-ad13dc9e-cc90-4651-b8d5-4532cbb5f605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:41:26.196947 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 803/1610] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 08:41:25 2022] PUT /identity/v3/projects/1cdf045fd98443afa186f8d3e2bf690e/users/b82a623527ad4280a353270fc3d51ba1/roles/44b64db484f24a46bf0b7ced46b519bf => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:41:26.198399 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b737467-bcc3-4a83-a0b8-0160b32939ac None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:26.198745 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b737467-bcc3-4a83-a0b8-0160b32939ac None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:26.199202 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b737467-bcc3-4a83-a0b8-0160b32939ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:26.224608 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-1b737467-bcc3-4a83-a0b8-0160b32939ac None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:41:26.226213 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-1b737467-bcc3-4a83-a0b8-0160b32939ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:41:26.233866 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 808/1611] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:41:26 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:41:26.233866 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da81054-72dd-4acb-a8a5-8d3a200decbf None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:26.233866 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da81054-72dd-4acb-a8a5-8d3a200decbf None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:26.233866 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da81054-72dd-4acb-a8a5-8d3a200decbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:26.255249 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:41:26.310753 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-7da81054-72dd-4acb-a8a5-8d3a200decbf None None] MFA Rules not processed for user `ff34a3644dc84bb49cca4de7f59c1d31`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:41:26.312734 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-7da81054-72dd-4acb-a8a5-8d3a200decbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:41:26.345600 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 804/1612] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:41:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:41:26.346685 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b737467-bcc3-4a83-a0b8-0160b32939ac None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:26.427047 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 809/1613] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:41:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:41:26.427970 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59862ba4-9337-4f19-b8b5-4d467607c0a4 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:26.428358 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59862ba4-9337-4f19-b8b5-4d467607c0a4 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:26.428727 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59862ba4-9337-4f19-b8b5-4d467607c0a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:26.432546 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b737467-bcc3-4a83-a0b8-0160b32939ac None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:26.437658 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baab960-e5af-47ba-900c-3d5ddbe392a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:26.445002 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baab960-e5af-47ba-900c-3d5ddbe392a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:26.445002 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baab960-e5af-47ba-900c-3d5ddbe392a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:26.445002 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baab960-e5af-47ba-900c-3d5ddbe392a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:26.450339 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-59862ba4-9337-4f19-b8b5-4d467607c0a4 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:41:26.451575 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-59862ba4-9337-4f19-b8b5-4d467607c0a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:41:26.454091 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baab960-e5af-47ba-900c-3d5ddbe392a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:26.455776 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baab960-e5af-47ba-900c-3d5ddbe392a2 service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:26.461590 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 810/1614] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:41:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:41:26.467001 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523bf3c4-0cb1-4dbb-b1f3-767f59e7b8b1 None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:26.467001 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523bf3c4-0cb1-4dbb-b1f3-767f59e7b8b1 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:26.467001 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523bf3c4-0cb1-4dbb-b1f3-767f59e7b8b1 None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:26.468267 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 805/1615] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:41:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:41:26.469337 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 811/1616] 10.222.0.57 () {54 vars in 962 bytes} [Wed Aug 10 08:41:26 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:41:26.470120 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523bf3c4-0cb1-4dbb-b1f3-767f59e7b8b1 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:26.473272 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baab960-e5af-47ba-900c-3d5ddbe392a2 service neutron] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:26.483353 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baab960-e5af-47ba-900c-3d5ddbe392a2 service neutron] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:26.487263 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c4be1f-d0a2-442a-9693-72beff7c4359 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:26.488004 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c4be1f-d0a2-442a-9693-72beff7c4359 service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:26.488156 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523bf3c4-0cb1-4dbb-b1f3-767f59e7b8b1 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:26.488411 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c4be1f-d0a2-442a-9693-72beff7c4359 service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:26.488936 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c4be1f-d0a2-442a-9693-72beff7c4359 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:26.495607 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f7a7f0-f8c2-40b4-ac2b-580b40c84845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:26.496222 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f7a7f0-f8c2-40b4-ac2b-580b40c84845 admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:26.496556 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f7a7f0-f8c2-40b4-ac2b-580b40c84845 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:26.496833 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f7a7f0-f8c2-40b4-ac2b-580b40c84845 admin admin] PATH_INFO: `/v3/projects` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:26.497519 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f7a7f0-f8c2-40b4-ac2b-580b40c84845 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:26.499647 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c4be1f-d0a2-442a-9693-72beff7c4359 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:26.500024 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f7a7f0-f8c2-40b4-ac2b-580b40c84845 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:26.506566 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c4be1f-d0a2-442a-9693-72beff7c4359 service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:26.521467 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 806/1617] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:41:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:41:26.526627 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f21006-a2a1-4b9c-a2eb-235ab3c2271a None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:26.526933 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f21006-a2a1-4b9c-a2eb-235ab3c2271a None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:26.527177 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f21006-a2a1-4b9c-a2eb-235ab3c2271a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:26.578196 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 812/1618] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 08:41:26 2022] POST /identity/v3/projects => generated 373 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:41:26.579330 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f7a7f0-f8c2-40b4-ac2b-580b40c84845 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:26.580770 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-71f21006-a2a1-4b9c-a2eb-235ab3c2271a None None] MFA Rules not processed for user `a943008193914f76a663d489bd73e873`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:41:26.583718 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-71f21006-a2a1-4b9c-a2eb-235ab3c2271a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:41:26.594062 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:41:26.672039 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f7a7f0-f8c2-40b4-ac2b-580b40c84845 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:26.685903 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d62781-b5b5-424a-b016-1813f55bb154 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:26.686588 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d62781-b5b5-424a-b016-1813f55bb154 service neutron] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:26.686869 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d62781-b5b5-424a-b016-1813f55bb154 service neutron] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:26.687516 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d62781-b5b5-424a-b016-1813f55bb154 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:26.744819 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 807/1619] 10.222.0.57 () {58 vars in 1059 bytes} [Wed Aug 10 08:41:26 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:41:26.747772 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f21006-a2a1-4b9c-a2eb-235ab3c2271a None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:26.791703 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d62781-b5b5-424a-b016-1813f55bb154 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:26.791703 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d62781-b5b5-424a-b016-1813f55bb154 service neutron] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:26.792499 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:41:26.826180 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 813/1620] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:41:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:41:26.828198 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d62781-b5b5-424a-b016-1813f55bb154 service neutron] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:26.845267 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f21006-a2a1-4b9c-a2eb-235ab3c2271a None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:26.849524 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d62781-b5b5-424a-b016-1813f55bb154 service neutron] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:26.852648 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004fb537-0a49-4829-9d44-61ae46765cfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:26.854573 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004fb537-0a49-4829-9d44-61ae46765cfd admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:26.854982 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004fb537-0a49-4829-9d44-61ae46765cfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:26.855319 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004fb537-0a49-4829-9d44-61ae46765cfd admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:26.856015 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004fb537-0a49-4829-9d44-61ae46765cfd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:26.858620 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004fb537-0a49-4829-9d44-61ae46765cfd admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:26.858788 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e34fc1-046d-4aa1-8f44-27961f82210f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a943008193914f76a663d489bd73e873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:26.859635 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e34fc1-046d-4aa1-8f44-27961f82210f service nova] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:26.860065 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e34fc1-046d-4aa1-8f44-27961f82210f service nova] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:26.860485 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e34fc1-046d-4aa1-8f44-27961f82210f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:26.933665 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e34fc1-046d-4aa1-8f44-27961f82210f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:26.935483 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e34fc1-046d-4aa1-8f44-27961f82210f service nova] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:26.946086 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 814/1621] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:41:26 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:41:26.958891 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 808/1622] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 08:41:26 2022] POST /identity/v3/users => generated 293 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:41:26.961297 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e34fc1-046d-4aa1-8f44-27961f82210f service nova] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:26.979083 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e34fc1-046d-4aa1-8f44-27961f82210f service nova] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:26.984348 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2624645-6164-4154-8b8b-42390b7f909f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:26.985177 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2624645-6164-4154-8b8b-42390b7f909f admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:26.985539 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2624645-6164-4154-8b8b-42390b7f909f admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:26.986141 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2624645-6164-4154-8b8b-42390b7f909f admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:26.987292 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2624645-6164-4154-8b8b-42390b7f909f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:26.989600 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2624645-6164-4154-8b8b-42390b7f909f admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:27.000981 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 815/1623] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:41:26 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:41:27.005223 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004fb537-0a49-4829-9d44-61ae46765cfd admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:27.021799 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004fb537-0a49-4829-9d44-61ae46765cfd admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:27.027956 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aec14d-a181-4020-9eb0-9cf0c7433a8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:27.028747 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aec14d-a181-4020-9eb0-9cf0c7433a8d admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:27.029133 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aec14d-a181-4020-9eb0-9cf0c7433a8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:27.029481 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aec14d-a181-4020-9eb0-9cf0c7433a8d admin admin] PATH_INFO: `/v3/projects/3190a143da434650ac8289326728c119/users/9b7f3cef0cef4a978dce997a17f6cd86/roles/ec3751692d9c4ecf81b4a71c936c46be` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:27.084090 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7aec14d-a181-4020-9eb0-9cf0c7433a8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3190a143da434650ac8289326728c119, user_id=9b7f3cef0cef4a978dce997a17f6cd86, role_id=ec3751692d9c4ecf81b4a71c936c46be)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:27.092969 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7aec14d-a181-4020-9eb0-9cf0c7433a8d admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:27.198047 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 809/1624] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 08:41:27 2022] PUT /identity/v3/projects/3190a143da434650ac8289326728c119/users/9b7f3cef0cef4a978dce997a17f6cd86/roles/ec3751692d9c4ecf81b4a71c936c46be => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:41:27.200351 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2624645-6164-4154-8b8b-42390b7f909f admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:27.217698 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2624645-6164-4154-8b8b-42390b7f909f admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:27.233679 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c41161-7ccc-41b1-933c-0c274710cf4c None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:27.234347 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c41161-7ccc-41b1-933c-0c274710cf4c None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:27.234875 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c41161-7ccc-41b1-933c-0c274710cf4c None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:27.237926 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 810/1625] 10.222.0.57 () {54 vars in 912 bytes} [Wed Aug 10 08:41:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 08:41:27.248842 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b973d5b-fd14-476a-bf87-22d1ceef3a3e None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:27.253671 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b973d5b-fd14-476a-bf87-22d1ceef3a3e None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:27.253671 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b973d5b-fd14-476a-bf87-22d1ceef3a3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:27.305825 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-2b973d5b-fd14-476a-bf87-22d1ceef3a3e None None] MFA Rules not processed for user `a943008193914f76a663d489bd73e873`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:41:27.316800 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-2b973d5b-fd14-476a-bf87-22d1ceef3a3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:41:27.317264 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:41:27.337755 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ef4772-5d40-4dd9-86be-1d00064c9bb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:27.338304 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ef4772-5d40-4dd9-86be-1d00064c9bb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:27.338591 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ef4772-5d40-4dd9-86be-1d00064c9bb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:27.338863 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ef4772-5d40-4dd9-86be-1d00064c9bb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:27.339548 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ef4772-5d40-4dd9-86be-1d00064c9bb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:27.342941 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ef4772-5d40-4dd9-86be-1d00064c9bb8 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:27.367768 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 816/1626] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:41:27 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:41:27.373844 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ef4772-5d40-4dd9-86be-1d00064c9bb8 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:27.408562 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ef4772-5d40-4dd9-86be-1d00064c9bb8 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:27.414536 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccf7d1f-2550-41b2-94e5-8122f78949dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:27.415225 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccf7d1f-2550-41b2-94e5-8122f78949dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:27.416070 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccf7d1f-2550-41b2-94e5-8122f78949dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:27.416361 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccf7d1f-2550-41b2-94e5-8122f78949dc admin admin] PATH_INFO: `/v3/projects/3190a143da434650ac8289326728c119/users/9b7f3cef0cef4a978dce997a17f6cd86/roles/ec3751692d9c4ecf81b4a71c936c46be` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:27.419490 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccf7d1f-2550-41b2-94e5-8122f78949dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=3190a143da434650ac8289326728c119, user_id=9b7f3cef0cef4a978dce997a17f6cd86, role_id=ec3751692d9c4ecf81b4a71c936c46be)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:27.423528 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccf7d1f-2550-41b2-94e5-8122f78949dc admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:27.457578 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 811/1628] 10.222.0.57 () {58 vars in 1009 bytes} [Wed Aug 10 08:41:27 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:41:27.459099 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd31531-261e-4033-a296-0cb8c71ed787 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:27.459550 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd31531-261e-4033-a296-0cb8c71ed787 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:27.459954 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd31531-261e-4033-a296-0cb8c71ed787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:27.468482 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 817/1628] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 08:41:27 2022] PUT /identity/v3/projects/3190a143da434650ac8289326728c119/users/9b7f3cef0cef4a978dce997a17f6cd86/roles/ec3751692d9c4ecf81b4a71c936c46be => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:41:27.469535 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccf7d1f-2550-41b2-94e5-8122f78949dc admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:27.501594 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-ebd31531-261e-4033-a296-0cb8c71ed787 None None] MFA Rules not processed for user `9b7f3cef0cef4a978dce997a17f6cd86`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:41:27.508573 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-ebd31531-261e-4033-a296-0cb8c71ed787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:41:27.586969 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccf7d1f-2550-41b2-94e5-8122f78949dc admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:27.596029 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a219c5a3-cdec-43e6-9d5d-e3d53cd59a77 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ba74e28ced4cbba857f54222dfbe41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:27.596029 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a219c5a3-cdec-43e6-9d5d-e3d53cd59a77 service cinder] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:27.596412 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a219c5a3-cdec-43e6-9d5d-e3d53cd59a77 service cinder] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:27.596628 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a219c5a3-cdec-43e6-9d5d-e3d53cd59a77 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:27.666617 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 812/1629] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:41:27 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:41:27.675764 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd31531-261e-4033-a296-0cb8c71ed787 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:27.690134 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd31531-261e-4033-a296-0cb8c71ed787 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:27.703453 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8522c9-e1ec-4d9e-a845-102da8873b64 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ba74e28ced4cbba857f54222dfbe41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:27.705013 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8522c9-e1ec-4d9e-a845-102da8873b64 service cinder] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:27.705013 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8522c9-e1ec-4d9e-a845-102da8873b64 service cinder] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:27.705013 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8522c9-e1ec-4d9e-a845-102da8873b64 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:27.724293 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:41:27.729368 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8522c9-e1ec-4d9e-a845-102da8873b64 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:27.736767 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a219c5a3-cdec-43e6-9d5d-e3d53cd59a77 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:27.736952 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8522c9-e1ec-4d9e-a845-102da8873b64 service cinder] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:27.737253 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a219c5a3-cdec-43e6-9d5d-e3d53cd59a77 service cinder] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:27.750027 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 818/1630] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:41:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 08:41:27.759422 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 813/1631] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:41:27 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:41:28.843634 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a8bb49-0d73-4901-8f56-f34152ccab3b None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:28.844482 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a8bb49-0d73-4901-8f56-f34152ccab3b None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:28.850650 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a8bb49-0d73-4901-8f56-f34152ccab3b None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:28.854464 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 819/1632] 10.222.0.57 () {54 vars in 962 bytes} [Wed Aug 10 08:41:28 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:41:28.912372 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c4019e-0e0d-4ff1-9461-0e28679bfe69 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:28.913302 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c4019e-0e0d-4ff1-9461-0e28679bfe69 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:28.913931 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c4019e-0e0d-4ff1-9461-0e28679bfe69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:28.963601 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-12c4019e-0e0d-4ff1-9461-0e28679bfe69 None None] MFA Rules not processed for user `a943008193914f76a663d489bd73e873`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:41:28.969668 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-12c4019e-0e0d-4ff1-9461-0e28679bfe69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:41:29.004965 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 814/1633] 10.222.0.57 () {58 vars in 1059 bytes} [Wed Aug 10 08:41:28 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:41:29.008158 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b546ed-7748-4be7-befe-6fd6a3b5ce50 None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:29.008456 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b546ed-7748-4be7-befe-6fd6a3b5ce50 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:29.009005 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b546ed-7748-4be7-befe-6fd6a3b5ce50 None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:29.010769 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 820/1634] 10.222.0.57 () {54 vars in 912 bytes} [Wed Aug 10 08:41:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:41:29.023458 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1ccb45-1f8f-4af4-9a56-d72313a1a288 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:29.023458 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1ccb45-1f8f-4af4-9a56-d72313a1a288 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:29.023458 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1ccb45-1f8f-4af4-9a56-d72313a1a288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:29.147734 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-8c1ccb45-1f8f-4af4-9a56-d72313a1a288 None None] MFA Rules not processed for user `a943008193914f76a663d489bd73e873`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:41:29.147734 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-8c1ccb45-1f8f-4af4-9a56-d72313a1a288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:41:29.147734 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 815/1635] 10.222.0.57 () {58 vars in 1009 bytes} [Wed Aug 10 08:41:29 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 08:41:29.149192 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b546ed-7748-4be7-befe-6fd6a3b5ce50 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:29.149192 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b546ed-7748-4be7-befe-6fd6a3b5ce50 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:29.158262 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d067676-136a-4fcf-8187-5a41d647ff5d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ba74e28ced4cbba857f54222dfbe41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:29.161093 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d067676-136a-4fcf-8187-5a41d647ff5d service cinder] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:29.161093 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d067676-136a-4fcf-8187-5a41d647ff5d service cinder] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:29.161093 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d067676-136a-4fcf-8187-5a41d647ff5d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:29.187502 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1ccb45-1f8f-4af4-9a56-d72313a1a288 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:29.199296 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d067676-136a-4fcf-8187-5a41d647ff5d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:29.418761 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d067676-136a-4fcf-8187-5a41d647ff5d service cinder] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:29.418761 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 821/1636] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:41:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:41:29.467532 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1ccb45-1f8f-4af4-9a56-d72313a1a288 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:29.467532 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedbee6a-abe8-4c17-8094-691167bdfa45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:29.467532 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedbee6a-abe8-4c17-8094-691167bdfa45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:29.467532 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedbee6a-abe8-4c17-8094-691167bdfa45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:29.469474 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedbee6a-abe8-4c17-8094-691167bdfa45 admin admin] PATH_INFO: `/v3/users/ff34a3644dc84bb49cca4de7f59c1d31` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:29.469474 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedbee6a-abe8-4c17-8094-691167bdfa45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff34a3644dc84bb49cca4de7f59c1d31)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:29.469474 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: WARNING py.warnings [None req-bedbee6a-abe8-4c17-8094-691167bdfa45 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 08:41:29.469474 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: warnings.warn(msg) Aug 10 08:41:29.469474 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: Aug 10 08:41:29.469474 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedbee6a-abe8-4c17-8094-691167bdfa45 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:29.847191 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-bedbee6a-abe8-4c17-8094-691167bdfa45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff34a3644dc84bb49cca4de7f59c1d31'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:41:29.899879 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-bedbee6a-abe8-4c17-8094-691167bdfa45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff34a3644dc84bb49cca4de7f59c1d31'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:41:29.943106 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-bedbee6a-abe8-4c17-8094-691167bdfa45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff34a3644dc84bb49cca4de7f59c1d31'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:41:29.945002 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-bedbee6a-abe8-4c17-8094-691167bdfa45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff34a3644dc84bb49cca4de7f59c1d31'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:41:29.945948 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-bedbee6a-abe8-4c17-8094-691167bdfa45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff34a3644dc84bb49cca4de7f59c1d31'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:41:30.050437 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 816/1637] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 08:41:29 2022] DELETE /identity/v3/users/ff34a3644dc84bb49cca4de7f59c1d31 => generated 0 bytes in 865 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:41:30.053158 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19f3a5d-d55c-49ec-ae33-ae9958db7d4f None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:30.053633 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19f3a5d-d55c-49ec-ae33-ae9958db7d4f None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:30.053732 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19f3a5d-d55c-49ec-ae33-ae9958db7d4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:30.122143 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-a19f3a5d-d55c-49ec-ae33-ae9958db7d4f None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:41:30.128974 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-a19f3a5d-d55c-49ec-ae33-ae9958db7d4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:41:30.302972 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:41:30.326788 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 822/1638] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:41:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:41:30.485192 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedbee6a-abe8-4c17-8094-691167bdfa45 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:30.486713 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-bedbee6a-abe8-4c17-8094-691167bdfa45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:41:30.579738 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedbee6a-abe8-4c17-8094-691167bdfa45 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:30.587172 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a132b350-dfe2-4403-ad8d-8611b82c26b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:30.587848 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a132b350-dfe2-4403-ad8d-8611b82c26b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:30.588129 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a132b350-dfe2-4403-ad8d-8611b82c26b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:30.588452 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a132b350-dfe2-4403-ad8d-8611b82c26b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:30.604560 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a132b350-dfe2-4403-ad8d-8611b82c26b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:30.606654 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a132b350-dfe2-4403-ad8d-8611b82c26b6 service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:30.625196 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 817/1639] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:41:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:41:31.845135 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19f3a5d-d55c-49ec-ae33-ae9958db7d4f None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:31.848549 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-a19f3a5d-d55c-49ec-ae33-ae9958db7d4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:41:31.884264 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19f3a5d-d55c-49ec-ae33-ae9958db7d4f None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:31.891944 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e241f39e-4704-4920-9cc0-1e14c0329943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:31.892843 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e241f39e-4704-4920-9cc0-1e14c0329943 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:31.894143 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e241f39e-4704-4920-9cc0-1e14c0329943 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:31.894553 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e241f39e-4704-4920-9cc0-1e14c0329943 admin admin] PATH_INFO: `/v3/projects/dd8d68233ef74767a19a50a277a495ad` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:31.896276 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e241f39e-4704-4920-9cc0-1e14c0329943 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd8d68233ef74767a19a50a277a495ad)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:31.897521 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: WARNING py.warnings [None req-e241f39e-4704-4920-9cc0-1e14c0329943 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 08:41:31.897521 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: warnings.warn(msg) Aug 10 08:41:31.897521 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: Aug 10 08:41:31.899156 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e241f39e-4704-4920-9cc0-1e14c0329943 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:32.109880 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-e241f39e-4704-4920-9cc0-1e14c0329943 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd8d68233ef74767a19a50a277a495ad', 'request_id': 'req-e241f39e-4704-4920-9cc0-1e14c0329943'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:41:32.182172 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-e241f39e-4704-4920-9cc0-1e14c0329943 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd8d68233ef74767a19a50a277a495ad', 'request_id': 'req-e241f39e-4704-4920-9cc0-1e14c0329943'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:41:32.290493 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-e241f39e-4704-4920-9cc0-1e14c0329943 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd8d68233ef74767a19a50a277a495ad', 'request_id': 'req-e241f39e-4704-4920-9cc0-1e14c0329943'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:41:32.329007 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-e241f39e-4704-4920-9cc0-1e14c0329943 admin admin] The token cache is being invalidate because project dd8d68233ef74767a19a50a277a495ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 08:41:32.329312 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-e241f39e-4704-4920-9cc0-1e14c0329943 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:41:32.330035 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-e241f39e-4704-4920-9cc0-1e14c0329943 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:41:32.337664 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 823/1640] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 08:41:31 2022] DELETE /identity/v3/projects/dd8d68233ef74767a19a50a277a495ad => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:41:32.338846 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a132b350-dfe2-4403-ad8d-8611b82c26b6 service neutron] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:32.347209 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:41:32.355316 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a132b350-dfe2-4403-ad8d-8611b82c26b6 service neutron] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:32.571456 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c5abab-23aa-4920-bb00-c042b27def08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:32.572026 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c5abab-23aa-4920-bb00-c042b27def08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:32.572302 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c5abab-23aa-4920-bb00-c042b27def08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:32.572569 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c5abab-23aa-4920-bb00-c042b27def08 admin admin] PATH_INFO: `/v3/users/9b7f3cef0cef4a978dce997a17f6cd86` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:32.573799 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c5abab-23aa-4920-bb00-c042b27def08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b7f3cef0cef4a978dce997a17f6cd86)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:32.575926 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c5abab-23aa-4920-bb00-c042b27def08 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:32.880320 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-15c5abab-23aa-4920-bb00-c042b27def08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b7f3cef0cef4a978dce997a17f6cd86'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:41:32.913243 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-15c5abab-23aa-4920-bb00-c042b27def08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b7f3cef0cef4a978dce997a17f6cd86'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:41:32.925374 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-15c5abab-23aa-4920-bb00-c042b27def08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b7f3cef0cef4a978dce997a17f6cd86'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:41:32.927664 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-15c5abab-23aa-4920-bb00-c042b27def08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b7f3cef0cef4a978dce997a17f6cd86'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:41:32.931514 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-15c5abab-23aa-4920-bb00-c042b27def08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b7f3cef0cef4a978dce997a17f6cd86'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:41:33.031652 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 818/1641] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 08:41:32 2022] DELETE /identity/v3/users/9b7f3cef0cef4a978dce997a17f6cd86 => generated 0 bytes in 693 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 08:41:34.805117 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e241f39e-4704-4920-9cc0-1e14c0329943 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:34.807324 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-e241f39e-4704-4920-9cc0-1e14c0329943 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:41:34.831857 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e241f39e-4704-4920-9cc0-1e14c0329943 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:34.969919 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc1d62e-8fd3-4035-8004-efbc62a93131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:34.970878 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc1d62e-8fd3-4035-8004-efbc62a93131 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:34.975200 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc1d62e-8fd3-4035-8004-efbc62a93131 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:34.975687 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc1d62e-8fd3-4035-8004-efbc62a93131 admin admin] PATH_INFO: `/v3/projects/3190a143da434650ac8289326728c119` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:34.980474 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc1d62e-8fd3-4035-8004-efbc62a93131 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3190a143da434650ac8289326728c119)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:34.984977 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc1d62e-8fd3-4035-8004-efbc62a93131 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:35.197652 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-6cc1d62e-8fd3-4035-8004-efbc62a93131 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3190a143da434650ac8289326728c119', 'request_id': 'req-6cc1d62e-8fd3-4035-8004-efbc62a93131'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:41:35.216961 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-6cc1d62e-8fd3-4035-8004-efbc62a93131 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3190a143da434650ac8289326728c119', 'request_id': 'req-6cc1d62e-8fd3-4035-8004-efbc62a93131'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:41:35.285491 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-6cc1d62e-8fd3-4035-8004-efbc62a93131 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3190a143da434650ac8289326728c119', 'request_id': 'req-6cc1d62e-8fd3-4035-8004-efbc62a93131'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:41:35.337282 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-6cc1d62e-8fd3-4035-8004-efbc62a93131 admin admin] The token cache is being invalidate because project 3190a143da434650ac8289326728c119 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 08:41:35.337736 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-6cc1d62e-8fd3-4035-8004-efbc62a93131 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:41:35.338677 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-6cc1d62e-8fd3-4035-8004-efbc62a93131 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:41:35.340909 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 824/1642] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 08:41:34 2022] DELETE /identity/v3/projects/3190a143da434650ac8289326728c119 => generated 0 bytes in 536 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:41:35.346050 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d8b3a7-b22f-457f-bac5-3c9cababf947 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:35.347420 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d8b3a7-b22f-457f-bac5-3c9cababf947 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:35.347831 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d8b3a7-b22f-457f-bac5-3c9cababf947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:35.395900 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-b2d8b3a7-b22f-457f-bac5-3c9cababf947 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:41:35.406782 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-b2d8b3a7-b22f-457f-bac5-3c9cababf947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:41:35.578507 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 819/1643] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:41:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:41:35.587816 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9c5e6f-bb52-410f-9dff-c666211fc318 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:35.588342 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9c5e6f-bb52-410f-9dff-c666211fc318 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:35.588839 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9c5e6f-bb52-410f-9dff-c666211fc318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:35.614636 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-de9c5e6f-bb52-410f-9dff-c666211fc318 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:41:35.616843 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-de9c5e6f-bb52-410f-9dff-c666211fc318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:41:35.629147 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 825/1644] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:41:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:41:35.634587 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d8b3a7-b22f-457f-bac5-3c9cababf947 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:35.658309 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d8b3a7-b22f-457f-bac5-3c9cababf947 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:35.670484 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a041d3d-4e9a-422a-91c0-8ebca966ac30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:35.671126 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a041d3d-4e9a-422a-91c0-8ebca966ac30 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:35.671410 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a041d3d-4e9a-422a-91c0-8ebca966ac30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:35.671954 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a041d3d-4e9a-422a-91c0-8ebca966ac30 admin admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:35.672964 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a041d3d-4e9a-422a-91c0-8ebca966ac30 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:41:35.673606 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a041d3d-4e9a-422a-91c0-8ebca966ac30 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:35.675575 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a041d3d-4e9a-422a-91c0-8ebca966ac30 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:35.693184 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 820/1645] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Aug 10 08:41:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:41:35.696729 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9c5e6f-bb52-410f-9dff-c666211fc318 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:35.703546 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:41:35.718882 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9c5e6f-bb52-410f-9dff-c666211fc318 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:35.734485 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881dbd7b-3de5-4f34-abbf-443d6208e94c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:35.735125 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881dbd7b-3de5-4f34-abbf-443d6208e94c admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:35.739173 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881dbd7b-3de5-4f34-abbf-443d6208e94c admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:35.739493 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881dbd7b-3de5-4f34-abbf-443d6208e94c admin admin] PATH_INFO: `/v3/projects` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:35.740177 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881dbd7b-3de5-4f34-abbf-443d6208e94c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:35.752713 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881dbd7b-3de5-4f34-abbf-443d6208e94c admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:35.881136 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 826/1646] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 08:41:35 2022] POST /identity/v3/projects => generated 357 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:41:35.886083 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a041d3d-4e9a-422a-91c0-8ebca966ac30 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:35.901592 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a041d3d-4e9a-422a-91c0-8ebca966ac30 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:35.914132 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:41:36.018103 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a4d2c8-90c0-46c2-927f-e31c63445aca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:36.026720 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a4d2c8-90c0-46c2-927f-e31c63445aca admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:36.027784 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a4d2c8-90c0-46c2-927f-e31c63445aca admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:36.029750 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a4d2c8-90c0-46c2-927f-e31c63445aca admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:36.030708 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a4d2c8-90c0-46c2-927f-e31c63445aca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:36.033074 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a4d2c8-90c0-46c2-927f-e31c63445aca admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:36.142783 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881dbd7b-3de5-4f34-abbf-443d6208e94c admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:36.142976 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 821/1647] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 08:41:35 2022] POST /identity/v3/users => generated 279 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:41:36.165394 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881dbd7b-3de5-4f34-abbf-443d6208e94c admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:36.179958 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134cfcb5-9cd6-4388-9e44-f908b9afea99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:36.179958 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134cfcb5-9cd6-4388-9e44-f908b9afea99 admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:36.179958 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134cfcb5-9cd6-4388-9e44-f908b9afea99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:36.179958 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134cfcb5-9cd6-4388-9e44-f908b9afea99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:36.183792 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134cfcb5-9cd6-4388-9e44-f908b9afea99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:36.187284 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134cfcb5-9cd6-4388-9e44-f908b9afea99 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:36.219935 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 827/1648] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:41:36 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:41:36.225591 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a4d2c8-90c0-46c2-927f-e31c63445aca admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:36.237191 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a4d2c8-90c0-46c2-927f-e31c63445aca admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:36.252441 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6098b9a-59f6-40ba-9ab2-d11665c01890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:36.253632 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6098b9a-59f6-40ba-9ab2-d11665c01890 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:36.253928 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6098b9a-59f6-40ba-9ab2-d11665c01890 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:36.254198 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6098b9a-59f6-40ba-9ab2-d11665c01890 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:36.262478 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6098b9a-59f6-40ba-9ab2-d11665c01890 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:36.264177 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6098b9a-59f6-40ba-9ab2-d11665c01890 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:36.310525 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134cfcb5-9cd6-4388-9e44-f908b9afea99 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:36.318847 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 822/1649] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:41:36 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:41:36.322500 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134cfcb5-9cd6-4388-9e44-f908b9afea99 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:36.333685 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2969b1ba-dbe3-49a4-8733-4e955fe100e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:36.334866 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2969b1ba-dbe3-49a4-8733-4e955fe100e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:36.334866 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2969b1ba-dbe3-49a4-8733-4e955fe100e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:36.335043 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2969b1ba-dbe3-49a4-8733-4e955fe100e0 admin admin] PATH_INFO: `/v3/projects/5c0b83c154c54e7980459d81591f09ee/users/ecb975b02b4447479aa36f7ccc9c474a/roles/44b64db484f24a46bf0b7ced46b519bf` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:36.369632 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2969b1ba-dbe3-49a4-8733-4e955fe100e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c0b83c154c54e7980459d81591f09ee, user_id=ecb975b02b4447479aa36f7ccc9c474a, role_id=44b64db484f24a46bf0b7ced46b519bf)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:36.372371 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2969b1ba-dbe3-49a4-8733-4e955fe100e0 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:36.469267 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca281f8-b1df-42dd-92e9-c94f930e60a0 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:36.469795 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca281f8-b1df-42dd-92e9-c94f930e60a0 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:36.469795 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca281f8-b1df-42dd-92e9-c94f930e60a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:36.472675 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 828/1650] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 08:41:36 2022] PUT /identity/v3/projects/5c0b83c154c54e7980459d81591f09ee/users/ecb975b02b4447479aa36f7ccc9c474a/roles/44b64db484f24a46bf0b7ced46b519bf => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:41:36.531587 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-0ca281f8-b1df-42dd-92e9-c94f930e60a0 None None] MFA Rules not processed for user `ecb975b02b4447479aa36f7ccc9c474a`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:41:36.534645 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-0ca281f8-b1df-42dd-92e9-c94f930e60a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:41:36.711611 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 823/1651] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:41:36 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:41:36.721052 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2969b1ba-dbe3-49a4-8733-4e955fe100e0 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:36.904592 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2969b1ba-dbe3-49a4-8733-4e955fe100e0 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:36.918022 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebe545a-3f1b-4c17-9f7e-256891a6837e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:36.918744 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebe545a-3f1b-4c17-9f7e-256891a6837e admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:36.919033 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebe545a-3f1b-4c17-9f7e-256891a6837e admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:36.919311 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebe545a-3f1b-4c17-9f7e-256891a6837e admin admin] PATH_INFO: `/v3/projects` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:36.925485 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebe545a-3f1b-4c17-9f7e-256891a6837e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:36.931518 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebe545a-3f1b-4c17-9f7e-256891a6837e admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:37.036293 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 829/1652] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 08:41:36 2022] POST /identity/v3/projects => generated 355 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:41:37.040844 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca281f8-b1df-42dd-92e9-c94f930e60a0 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:37.062795 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca281f8-b1df-42dd-92e9-c94f930e60a0 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:37.239424 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9963e3d-ea7d-4554-8693-d6aad97129ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:37.240069 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9963e3d-ea7d-4554-8693-d6aad97129ad admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:37.240349 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9963e3d-ea7d-4554-8693-d6aad97129ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:37.240675 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9963e3d-ea7d-4554-8693-d6aad97129ad admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:37.241417 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9963e3d-ea7d-4554-8693-d6aad97129ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:37.243539 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9963e3d-ea7d-4554-8693-d6aad97129ad admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:37.348545 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 824/1653] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 08:41:37 2022] POST /identity/v3/users => generated 284 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:41:37.352324 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebe545a-3f1b-4c17-9f7e-256891a6837e admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:37.369374 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebe545a-3f1b-4c17-9f7e-256891a6837e admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:37.387245 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f30ef6-9eac-46c9-9270-4a6b9c2aaacc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:37.390467 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f30ef6-9eac-46c9-9270-4a6b9c2aaacc admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:37.390467 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f30ef6-9eac-46c9-9270-4a6b9c2aaacc admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:37.390467 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f30ef6-9eac-46c9-9270-4a6b9c2aaacc admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:37.390467 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f30ef6-9eac-46c9-9270-4a6b9c2aaacc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:37.392199 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f30ef6-9eac-46c9-9270-4a6b9c2aaacc admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:37.441796 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 830/1654] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:41:37 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:41:37.446202 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9963e3d-ea7d-4554-8693-d6aad97129ad admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:37.460641 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9963e3d-ea7d-4554-8693-d6aad97129ad admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:37.468313 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3c3f6c-5eb1-486e-a9c8-8696a8f1fb43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:37.469299 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3c3f6c-5eb1-486e-a9c8-8696a8f1fb43 admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:37.471703 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3c3f6c-5eb1-486e-a9c8-8696a8f1fb43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:37.472239 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3c3f6c-5eb1-486e-a9c8-8696a8f1fb43 admin admin] PATH_INFO: `/v3/projects/29b4622415af4edfaedfec8dec05ac3e/users/282055067f0143b488116d323c475c7f/roles/ec3751692d9c4ecf81b4a71c936c46be` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:37.476774 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:41:37.492903 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3c3f6c-5eb1-486e-a9c8-8696a8f1fb43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29b4622415af4edfaedfec8dec05ac3e, user_id=282055067f0143b488116d323c475c7f, role_id=ec3751692d9c4ecf81b4a71c936c46be)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:37.498058 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3c3f6c-5eb1-486e-a9c8-8696a8f1fb43 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:37.574007 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 825/1655] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 08:41:37 2022] PUT /identity/v3/projects/29b4622415af4edfaedfec8dec05ac3e/users/282055067f0143b488116d323c475c7f/roles/ec3751692d9c4ecf81b4a71c936c46be => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:41:37.578009 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f30ef6-9eac-46c9-9270-4a6b9c2aaacc admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:37.591934 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f30ef6-9eac-46c9-9270-4a6b9c2aaacc admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:37.696873 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc25b091-eda2-4536-9795-57f487ef8cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:37.697564 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc25b091-eda2-4536-9795-57f487ef8cc9 admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:37.697853 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc25b091-eda2-4536-9795-57f487ef8cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:37.698186 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc25b091-eda2-4536-9795-57f487ef8cc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:37.698863 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc25b091-eda2-4536-9795-57f487ef8cc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:37.705313 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc25b091-eda2-4536-9795-57f487ef8cc9 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:37.722232 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 831/1656] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:41:37 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:41:37.726157 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3c3f6c-5eb1-486e-a9c8-8696a8f1fb43 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:37.742120 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3c3f6c-5eb1-486e-a9c8-8696a8f1fb43 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:37.754106 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a3c653-1da1-436a-9141-a1b917783f73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:37.754876 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a3c653-1da1-436a-9141-a1b917783f73 admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:37.755353 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a3c653-1da1-436a-9141-a1b917783f73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:37.755842 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a3c653-1da1-436a-9141-a1b917783f73 admin admin] PATH_INFO: `/v3/projects/29b4622415af4edfaedfec8dec05ac3e/users/282055067f0143b488116d323c475c7f/roles/ec3751692d9c4ecf81b4a71c936c46be` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:37.758529 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a3c653-1da1-436a-9141-a1b917783f73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29b4622415af4edfaedfec8dec05ac3e, user_id=282055067f0143b488116d323c475c7f, role_id=ec3751692d9c4ecf81b4a71c936c46be)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:37.770927 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a3c653-1da1-436a-9141-a1b917783f73 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:37.786187 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 826/1657] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 08:41:37 2022] PUT /identity/v3/projects/29b4622415af4edfaedfec8dec05ac3e/users/282055067f0143b488116d323c475c7f/roles/ec3751692d9c4ecf81b4a71c936c46be => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:41:37.807582 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc63918-d10a-4b6b-8e0d-a8609a91f613 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:37.812336 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc63918-d10a-4b6b-8e0d-a8609a91f613 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:37.812883 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc63918-d10a-4b6b-8e0d-a8609a91f613 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:37.966867 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-7cc63918-d10a-4b6b-8e0d-a8609a91f613 None None] MFA Rules not processed for user `282055067f0143b488116d323c475c7f`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:41:37.968095 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-7cc63918-d10a-4b6b-8e0d-a8609a91f613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:41:37.986622 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:41:38.072004 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394f8959-eaac-4bc8-ba8a-24b5b89ece4b None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:38.072534 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394f8959-eaac-4bc8-ba8a-24b5b89ece4b None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:38.073028 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394f8959-eaac-4bc8-ba8a-24b5b89ece4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:38.133509 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 832/1658] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:41:37 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 329 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:41:38.156567 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc63918-d10a-4b6b-8e0d-a8609a91f613 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:38.161654 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-7cc63918-d10a-4b6b-8e0d-a8609a91f613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:41:38.187101 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-394f8959-eaac-4bc8-ba8a-24b5b89ece4b None None] MFA Rules not processed for user `258ec0f975eb47738f4dac356c0578e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:41:38.191851 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-394f8959-eaac-4bc8-ba8a-24b5b89ece4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:41:38.278583 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc63918-d10a-4b6b-8e0d-a8609a91f613 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:38.288184 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9711ea87-d5b2-43bd-b11d-42555d6f94fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ba74e28ced4cbba857f54222dfbe41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:38.289158 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9711ea87-d5b2-43bd-b11d-42555d6f94fc service cinder] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:38.289158 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9711ea87-d5b2-43bd-b11d-42555d6f94fc service cinder] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:38.289158 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9711ea87-d5b2-43bd-b11d-42555d6f94fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:38.317691 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9711ea87-d5b2-43bd-b11d-42555d6f94fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:38.319186 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9711ea87-d5b2-43bd-b11d-42555d6f94fc service cinder] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:38.357619 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 833/1659] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:41:38 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:41:38.638318 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 827/1660] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:41:38 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 350 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:41:38.639333 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9711ea87-d5b2-43bd-b11d-42555d6f94fc service cinder] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:38.639333 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-9711ea87-d5b2-43bd-b11d-42555d6f94fc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:41:38.775152 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9711ea87-d5b2-43bd-b11d-42555d6f94fc service cinder] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:38.775411 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394f8959-eaac-4bc8-ba8a-24b5b89ece4b None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:38.800888 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394f8959-eaac-4bc8-ba8a-24b5b89ece4b None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:38.803641 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1913172-fee2-4aac-ae2b-052933d1cc62 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a943008193914f76a663d489bd73e873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:38.804583 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1913172-fee2-4aac-ae2b-052933d1cc62 service nova] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:38.806377 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1913172-fee2-4aac-ae2b-052933d1cc62 service nova] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:38.807526 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1913172-fee2-4aac-ae2b-052933d1cc62 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:38.816671 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4c2ea4-6a6b-46c3-8eb5-c493830fe631 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ba74e28ced4cbba857f54222dfbe41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:38.821418 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4c2ea4-6a6b-46c3-8eb5-c493830fe631 service cinder] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:38.823885 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4c2ea4-6a6b-46c3-8eb5-c493830fe631 service cinder] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:38.824278 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4c2ea4-6a6b-46c3-8eb5-c493830fe631 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:38.842509 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1913172-fee2-4aac-ae2b-052933d1cc62 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:38.860439 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1913172-fee2-4aac-ae2b-052933d1cc62 service nova] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:38.899283 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 834/1661] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:41:38 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:41:39.040979 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4c2ea4-6a6b-46c3-8eb5-c493830fe631 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:39.045541 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4c2ea4-6a6b-46c3-8eb5-c493830fe631 service cinder] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:39.094916 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 828/1662] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:41:38 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 08:41:41.462142 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1913172-fee2-4aac-ae2b-052933d1cc62 service nova] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:41.462602 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-d1913172-fee2-4aac-ae2b-052933d1cc62 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:41:41.601703 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:41:41.704683 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1913172-fee2-4aac-ae2b-052933d1cc62 service nova] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:41.719603 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480a5f53-0f72-4ac2-8ca1-91e99e33f3e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:41.721016 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480a5f53-0f72-4ac2-8ca1-91e99e33f3e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:41.722014 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480a5f53-0f72-4ac2-8ca1-91e99e33f3e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:41.722459 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480a5f53-0f72-4ac2-8ca1-91e99e33f3e8 admin admin] PATH_INFO: `/v3/users/8622a62d2fd4484c9444bc4348cecdfe` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:41.723808 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480a5f53-0f72-4ac2-8ca1-91e99e33f3e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8622a62d2fd4484c9444bc4348cecdfe)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:41.726050 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480a5f53-0f72-4ac2-8ca1-91e99e33f3e8 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:41.998952 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-480a5f53-0f72-4ac2-8ca1-91e99e33f3e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8622a62d2fd4484c9444bc4348cecdfe'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:41:42.015355 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e01e4f-60ac-4eb8-8d95-58264c78e5b3 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:42.016382 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e01e4f-60ac-4eb8-8d95-58264c78e5b3 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:42.016800 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e01e4f-60ac-4eb8-8d95-58264c78e5b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:42.039827 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-480a5f53-0f72-4ac2-8ca1-91e99e33f3e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8622a62d2fd4484c9444bc4348cecdfe'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:41:42.062021 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-480a5f53-0f72-4ac2-8ca1-91e99e33f3e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8622a62d2fd4484c9444bc4348cecdfe'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:41:42.085778 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-e3e01e4f-60ac-4eb8-8d95-58264c78e5b3 None None] MFA Rules not processed for user `9797df3e05944552b741f53773abf80a`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:41:42.087077 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-e3e01e4f-60ac-4eb8-8d95-58264c78e5b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:41:42.135458 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-480a5f53-0f72-4ac2-8ca1-91e99e33f3e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8622a62d2fd4484c9444bc4348cecdfe'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:41:42.136606 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-480a5f53-0f72-4ac2-8ca1-91e99e33f3e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8622a62d2fd4484c9444bc4348cecdfe'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:41:42.155125 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 835/1663] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 08:41:41 2022] DELETE /identity/v3/users/8622a62d2fd4484c9444bc4348cecdfe => generated 0 bytes in 698 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:41:42.160849 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a56484-0a6a-43d8-ae47-85c540b00181 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:42.161684 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a56484-0a6a-43d8-ae47-85c540b00181 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:42.161988 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a56484-0a6a-43d8-ae47-85c540b00181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:42.213673 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-f3a56484-0a6a-43d8-ae47-85c540b00181 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:41:42.214857 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-f3a56484-0a6a-43d8-ae47-85c540b00181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:41:42.234776 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:41:42.288495 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 829/1664] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:41:42 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 08:41:42.309279 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e01e4f-60ac-4eb8-8d95-58264c78e5b3 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:42.327184 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-e3e01e4f-60ac-4eb8-8d95-58264c78e5b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:41:42.375962 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e01e4f-60ac-4eb8-8d95-58264c78e5b3 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:42.475571 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 836/1665] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:41:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:41:42.490087 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a56484-0a6a-43d8-ae47-85c540b00181 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:42.491913 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-f3a56484-0a6a-43d8-ae47-85c540b00181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:41:42.570709 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6282a1b8-61e6-4e5e-8fea-af6bdbf44297 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a943008193914f76a663d489bd73e873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:42.572191 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6282a1b8-61e6-4e5e-8fea-af6bdbf44297 service nova] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:42.572615 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6282a1b8-61e6-4e5e-8fea-af6bdbf44297 service nova] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:42.573054 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6282a1b8-61e6-4e5e-8fea-af6bdbf44297 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:42.707885 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a56484-0a6a-43d8-ae47-85c540b00181 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:42.724224 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d35b41e-a680-470e-9894-3c3b9ae35612 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:42.727385 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d35b41e-a680-470e-9894-3c3b9ae35612 service neutron] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:42.727690 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d35b41e-a680-470e-9894-3c3b9ae35612 service neutron] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:42.729170 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d35b41e-a680-470e-9894-3c3b9ae35612 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:42.766021 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6282a1b8-61e6-4e5e-8fea-af6bdbf44297 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:42.767908 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6282a1b8-61e6-4e5e-8fea-af6bdbf44297 service nova] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:42.772310 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:41:42.780131 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d35b41e-a680-470e-9894-3c3b9ae35612 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:42.787259 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d35b41e-a680-470e-9894-3c3b9ae35612 service neutron] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:42.812353 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 830/1666] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:41:42 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:41:42.845428 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 837/1667] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:41:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 08:41:43.588787 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c24bda-222e-426d-b450-e839b76b0286 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:43.589362 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c24bda-222e-426d-b450-e839b76b0286 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:43.589362 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c24bda-222e-426d-b450-e839b76b0286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:43.672720 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-88c24bda-222e-426d-b450-e839b76b0286 None None] MFA Rules not processed for user `3bf116fade674838b9219e15dd20eda2`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:41:43.673748 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-88c24bda-222e-426d-b450-e839b76b0286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:41:43.836909 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 831/1668] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:41:43 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:41:43.852403 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d35b41e-a680-470e-9894-3c3b9ae35612 service neutron] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:43.855367 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-0d35b41e-a680-470e-9894-3c3b9ae35612 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:41:43.913113 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d35b41e-a680-470e-9894-3c3b9ae35612 service neutron] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:43.927629 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823edd0a-5682-4056-a61e-bb6bc4992523 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a943008193914f76a663d489bd73e873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:43.933254 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823edd0a-5682-4056-a61e-bb6bc4992523 service nova] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:43.934569 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823edd0a-5682-4056-a61e-bb6bc4992523 service nova] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:43.935365 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823edd0a-5682-4056-a61e-bb6bc4992523 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:43.949936 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823edd0a-5682-4056-a61e-bb6bc4992523 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:43.951805 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823edd0a-5682-4056-a61e-bb6bc4992523 service nova] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:43.987985 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 838/1669] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:41:43 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:41:44.447301 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c24bda-222e-426d-b450-e839b76b0286 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:44.449289 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-88c24bda-222e-426d-b450-e839b76b0286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:41:44.474866 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c24bda-222e-426d-b450-e839b76b0286 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:44.481406 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a73202-7afc-4e9b-a4a9-30c78accff9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:44.482134 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a73202-7afc-4e9b-a4a9-30c78accff9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:44.482487 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a73202-7afc-4e9b-a4a9-30c78accff9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:44.483016 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a73202-7afc-4e9b-a4a9-30c78accff9d admin admin] PATH_INFO: `/v3/projects/72624a2133804ace9905a30f82bf1148` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:44.485232 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a73202-7afc-4e9b-a4a9-30c78accff9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=72624a2133804ace9905a30f82bf1148)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:44.511970 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90edad1-115e-457f-b5e9-e86c02cfbe97 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:44.512387 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90edad1-115e-457f-b5e9-e86c02cfbe97 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:44.512661 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90edad1-115e-457f-b5e9-e86c02cfbe97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:44.526640 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a73202-7afc-4e9b-a4a9-30c78accff9d admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:44.576566 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:41:44.597183 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-e90edad1-115e-457f-b5e9-e86c02cfbe97 None None] MFA Rules not processed for user `b82a623527ad4280a353270fc3d51ba1`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:41:44.599670 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-e90edad1-115e-457f-b5e9-e86c02cfbe97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:41:44.714676 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 839/1670] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:41:44 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:41:44.716580 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d568501-0fee-4583-bdc8-191940e4088d None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:44.717700 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d568501-0fee-4583-bdc8-191940e4088d None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:44.718025 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-63a73202-7afc-4e9b-a4a9-30c78accff9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72624a2133804ace9905a30f82bf1148', 'request_id': 'req-63a73202-7afc-4e9b-a4a9-30c78accff9d'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:41:44.718687 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d568501-0fee-4583-bdc8-191940e4088d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:44.750518 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-6d568501-0fee-4583-bdc8-191940e4088d None None] MFA Rules not processed for user `b82a623527ad4280a353270fc3d51ba1`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:41:44.752008 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-6d568501-0fee-4583-bdc8-191940e4088d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:41:44.822494 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 840/1671] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:41:44 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:41:44.834015 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d568501-0fee-4583-bdc8-191940e4088d None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:44.845524 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-63a73202-7afc-4e9b-a4a9-30c78accff9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72624a2133804ace9905a30f82bf1148', 'request_id': 'req-63a73202-7afc-4e9b-a4a9-30c78accff9d'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:41:44.856531 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-63a73202-7afc-4e9b-a4a9-30c78accff9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72624a2133804ace9905a30f82bf1148', 'request_id': 'req-63a73202-7afc-4e9b-a4a9-30c78accff9d'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:41:44.900390 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-63a73202-7afc-4e9b-a4a9-30c78accff9d admin admin] The token cache is being invalidate because project 72624a2133804ace9905a30f82bf1148 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 08:41:44.900697 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-63a73202-7afc-4e9b-a4a9-30c78accff9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:41:44.901588 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-63a73202-7afc-4e9b-a4a9-30c78accff9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:41:44.903144 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 832/1672] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 08:41:44 2022] DELETE /identity/v3/projects/72624a2133804ace9905a30f82bf1148 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:41:44.909613 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe5f3c9-f8a0-4f2b-a2e5-b040ee7d755b None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:44.910247 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe5f3c9-f8a0-4f2b-a2e5-b040ee7d755b None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:44.910675 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe5f3c9-f8a0-4f2b-a2e5-b040ee7d755b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:44.978300 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-4fe5f3c9-f8a0-4f2b-a2e5-b040ee7d755b None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:41:44.988651 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-4fe5f3c9-f8a0-4f2b-a2e5-b040ee7d755b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:41:45.027440 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d568501-0fee-4583-bdc8-191940e4088d None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:45.051364 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e8d820-3460-4a52-a343-f3f0233ce8a0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a943008193914f76a663d489bd73e873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:45.052474 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e8d820-3460-4a52-a343-f3f0233ce8a0 service nova] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:45.052959 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e8d820-3460-4a52-a343-f3f0233ce8a0 service nova] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:45.056848 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e8d820-3460-4a52-a343-f3f0233ce8a0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:45.057617 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-d3e8d820-3460-4a52-a343-f3f0233ce8a0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:41:45.097039 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e8d820-3460-4a52-a343-f3f0233ce8a0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:45.097039 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e8d820-3460-4a52-a343-f3f0233ce8a0 service nova] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:45.121173 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:41:45.125803 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 841/1673] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:41:44 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 08:41:45.170970 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 833/1674] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:41:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:41:45.179241 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af0644-c2b8-42b7-b921-2a078d73b2c6 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:45.179459 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af0644-c2b8-42b7-b921-2a078d73b2c6 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:45.179649 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af0644-c2b8-42b7-b921-2a078d73b2c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:45.224077 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-00af0644-c2b8-42b7-b921-2a078d73b2c6 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:41:45.226904 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-00af0644-c2b8-42b7-b921-2a078d73b2c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:41:45.237649 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 842/1675] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:41:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:41:45.241633 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe5f3c9-f8a0-4f2b-a2e5-b040ee7d755b None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:45.260070 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe5f3c9-f8a0-4f2b-a2e5-b040ee7d755b None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:45.265278 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00af0644-c2b8-42b7-b921-2a078d73b2c6 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:45.270200 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-00af0644-c2b8-42b7-b921-2a078d73b2c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:41:45.279057 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32ad4f1-2666-48a5-bc5b-97bc679893b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:45.279704 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32ad4f1-2666-48a5-bc5b-97bc679893b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:45.279985 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32ad4f1-2666-48a5-bc5b-97bc679893b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:45.280255 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32ad4f1-2666-48a5-bc5b-97bc679893b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:45.280961 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32ad4f1-2666-48a5-bc5b-97bc679893b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:41:45.281256 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32ad4f1-2666-48a5-bc5b-97bc679893b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:45.282980 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32ad4f1-2666-48a5-bc5b-97bc679893b3 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:45.325452 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00af0644-c2b8-42b7-b921-2a078d73b2c6 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:45.343801 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20421ab6-7b73-4b73-9b79-c4806397afe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:45.347231 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20421ab6-7b73-4b73-9b79-c4806397afe8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:45.349090 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20421ab6-7b73-4b73-9b79-c4806397afe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:45.349389 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20421ab6-7b73-4b73-9b79-c4806397afe8 admin admin] PATH_INFO: `/v3/users/e43802beaa824a5cb00759664f5bd646` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:45.352342 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20421ab6-7b73-4b73-9b79-c4806397afe8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e43802beaa824a5cb00759664f5bd646)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:45.359101 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 834/1676] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Aug 10 08:41:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:41:45.361526 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32ad4f1-2666-48a5-bc5b-97bc679893b3 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:45.362974 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20421ab6-7b73-4b73-9b79-c4806397afe8 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:45.404420 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32ad4f1-2666-48a5-bc5b-97bc679893b3 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:45.413780 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81de116b-1688-499e-8674-2d0d9ca2a5e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:45.413780 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81de116b-1688-499e-8674-2d0d9ca2a5e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:45.413780 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81de116b-1688-499e-8674-2d0d9ca2a5e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:45.413780 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81de116b-1688-499e-8674-2d0d9ca2a5e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:45.414357 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81de116b-1688-499e-8674-2d0d9ca2a5e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:45.415198 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81de116b-1688-499e-8674-2d0d9ca2a5e1 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:45.570161 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 835/1677] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 08:41:45 2022] POST /identity/v3/projects => generated 381 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:41:45.573385 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81de116b-1688-499e-8674-2d0d9ca2a5e1 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:45.664400 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-20421ab6-7b73-4b73-9b79-c4806397afe8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e43802beaa824a5cb00759664f5bd646'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:41:45.705812 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-20421ab6-7b73-4b73-9b79-c4806397afe8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e43802beaa824a5cb00759664f5bd646'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:41:45.726789 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-20421ab6-7b73-4b73-9b79-c4806397afe8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e43802beaa824a5cb00759664f5bd646'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:41:45.741909 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81de116b-1688-499e-8674-2d0d9ca2a5e1 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:45.760698 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cabf51-a9ce-441a-8670-1e424f727385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:45.761565 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cabf51-a9ce-441a-8670-1e424f727385 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:45.762048 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cabf51-a9ce-441a-8670-1e424f727385 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:45.762667 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cabf51-a9ce-441a-8670-1e424f727385 admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:45.763523 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cabf51-a9ce-441a-8670-1e424f727385 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:45.765803 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cabf51-a9ce-441a-8670-1e424f727385 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:45.784516 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-20421ab6-7b73-4b73-9b79-c4806397afe8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e43802beaa824a5cb00759664f5bd646'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:41:45.785381 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-20421ab6-7b73-4b73-9b79-c4806397afe8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e43802beaa824a5cb00759664f5bd646'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:41:45.789455 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:41:45.813668 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 843/1678] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 08:41:45 2022] DELETE /identity/v3/users/e43802beaa824a5cb00759664f5bd646 => generated 0 bytes in 549 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 08:41:45.820252 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf02df0-c0d0-422c-a688-0105787b6c75 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:45.821615 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf02df0-c0d0-422c-a688-0105787b6c75 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:45.821615 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf02df0-c0d0-422c-a688-0105787b6c75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:45.860822 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-bdf02df0-c0d0-422c-a688-0105787b6c75 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:41:45.862558 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-bdf02df0-c0d0-422c-a688-0105787b6c75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:41:45.897426 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 836/1679] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 08:41:45 2022] POST /identity/v3/users => generated 291 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:41:45.899549 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cabf51-a9ce-441a-8670-1e424f727385 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:45.910023 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-30cabf51-a9ce-441a-8670-1e424f727385 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:41:45.954119 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cabf51-a9ce-441a-8670-1e424f727385 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:46.031615 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 844/1680] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:41:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:41:46.060039 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf02df0-c0d0-422c-a688-0105787b6c75 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:46.061741 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-bdf02df0-c0d0-422c-a688-0105787b6c75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:41:46.201824 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d70c7bf-88ad-4d52-a38d-8d91a6ccf698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:46.203264 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d70c7bf-88ad-4d52-a38d-8d91a6ccf698 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:46.203670 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d70c7bf-88ad-4d52-a38d-8d91a6ccf698 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:46.206909 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d70c7bf-88ad-4d52-a38d-8d91a6ccf698 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:46.207859 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d70c7bf-88ad-4d52-a38d-8d91a6ccf698 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:46.209676 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d70c7bf-88ad-4d52-a38d-8d91a6ccf698 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:46.240940 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 837/1681] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:41:45 2022] GET /identity/v3/roles => generated 1420 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:41:46.243787 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc765d9-facf-420f-a2ca-4f6e91b4035a None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:46.265798 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf02df0-c0d0-422c-a688-0105787b6c75 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:46.277108 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc765d9-facf-420f-a2ca-4f6e91b4035a None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:46.277659 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc765d9-facf-420f-a2ca-4f6e91b4035a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:46.289202 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900a617f-bf1e-4e61-b1e6-ea9d68a42863 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:46.289776 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900a617f-bf1e-4e61-b1e6-ea9d68a42863 service neutron] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:46.294549 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900a617f-bf1e-4e61-b1e6-ea9d68a42863 service neutron] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:46.294876 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900a617f-bf1e-4e61-b1e6-ea9d68a42863 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:46.334062 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900a617f-bf1e-4e61-b1e6-ea9d68a42863 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:46.336240 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900a617f-bf1e-4e61-b1e6-ea9d68a42863 service neutron] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:46.360133 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-7bc765d9-facf-420f-a2ca-4f6e91b4035a None None] MFA Rules not processed for user `4ec7ba61e65447a5a87b0f8fcc2b946c`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:41:46.362024 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-7bc765d9-facf-420f-a2ca-4f6e91b4035a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:41:46.385307 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 845/1682] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:41:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 08:41:46.386324 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900a617f-bf1e-4e61-b1e6-ea9d68a42863 service neutron] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:46.413948 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900a617f-bf1e-4e61-b1e6-ea9d68a42863 service neutron] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:46.432287 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e35783-1679-4cc2-93d4-e91ffe306fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:46.432940 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e35783-1679-4cc2-93d4-e91ffe306fb0 admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:46.433231 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e35783-1679-4cc2-93d4-e91ffe306fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:46.433644 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e35783-1679-4cc2-93d4-e91ffe306fb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:46.434311 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e35783-1679-4cc2-93d4-e91ffe306fb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:46.438558 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e35783-1679-4cc2-93d4-e91ffe306fb0 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:46.449533 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:41:46.482181 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 846/1683] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:41:46 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:41:46.483194 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e35783-1679-4cc2-93d4-e91ffe306fb0 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:46.507680 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 838/1684] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:41:46 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:41:46.511763 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e35783-1679-4cc2-93d4-e91ffe306fb0 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:46.520053 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6944c4ac-02b5-4b8c-a526-c74c7bb5e874 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:46.523561 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6944c4ac-02b5-4b8c-a526-c74c7bb5e874 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:46.523561 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6944c4ac-02b5-4b8c-a526-c74c7bb5e874 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:46.523750 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51930e87-ce10-4f8a-936a-28d7bc810b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:46.523871 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51930e87-ce10-4f8a-936a-28d7bc810b13 admin admin] REQUEST_METHOD: `PUT` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:46.524401 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51930e87-ce10-4f8a-936a-28d7bc810b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:46.524690 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51930e87-ce10-4f8a-936a-28d7bc810b13 admin admin] PATH_INFO: `/v3/projects/6acf0b5477e940c08b6b50122a4808bc/users/24b90fa3db904d76aea127f094d2fd9f/roles/44b64db484f24a46bf0b7ced46b519bf` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:46.569990 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-6944c4ac-02b5-4b8c-a526-c74c7bb5e874 None None] MFA Rules not processed for user `4ec7ba61e65447a5a87b0f8fcc2b946c`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:41:46.571651 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-6944c4ac-02b5-4b8c-a526-c74c7bb5e874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:41:46.583972 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51930e87-ce10-4f8a-936a-28d7bc810b13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6acf0b5477e940c08b6b50122a4808bc, user_id=24b90fa3db904d76aea127f094d2fd9f, role_id=44b64db484f24a46bf0b7ced46b519bf)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:46.589088 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51930e87-ce10-4f8a-936a-28d7bc810b13 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:46.590561 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 839/1685] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:41:46 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:41:46.600156 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6944c4ac-02b5-4b8c-a526-c74c7bb5e874 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:46.601772 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-6944c4ac-02b5-4b8c-a526-c74c7bb5e874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:41:46.698679 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 847/1686] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 08:41:46 2022] PUT /identity/v3/projects/6acf0b5477e940c08b6b50122a4808bc/users/24b90fa3db904d76aea127f094d2fd9f/roles/44b64db484f24a46bf0b7ced46b519bf => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:41:46.710591 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c908a-0e84-426d-b83e-19d255c60300 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:46.710918 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c908a-0e84-426d-b83e-19d255c60300 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:46.711208 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c908a-0e84-426d-b83e-19d255c60300 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:46.779660 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-e40c908a-0e84-426d-b83e-19d255c60300 None None] MFA Rules not processed for user `24b90fa3db904d76aea127f094d2fd9f`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:41:46.781839 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-e40c908a-0e84-426d-b83e-19d255c60300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:41:46.832393 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6944c4ac-02b5-4b8c-a526-c74c7bb5e874 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:46.852614 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:41:46.953974 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 848/1687] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:41:46 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:41:46.959632 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40c908a-0e84-426d-b83e-19d255c60300 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:46.985314 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76205879-91b5-43de-bdf7-b4f8678aef4c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a943008193914f76a663d489bd73e873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:46.986049 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76205879-91b5-43de-bdf7-b4f8678aef4c service nova] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:46.986221 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76205879-91b5-43de-bdf7-b4f8678aef4c service nova] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:46.991542 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76205879-91b5-43de-bdf7-b4f8678aef4c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:47.103338 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40c908a-0e84-426d-b83e-19d255c60300 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:47.116824 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca4e6f3-ab40-4acd-865f-e307b4b9ce06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:47.118913 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca4e6f3-ab40-4acd-865f-e307b4b9ce06 admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:47.119213 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca4e6f3-ab40-4acd-865f-e307b4b9ce06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:47.119491 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca4e6f3-ab40-4acd-865f-e307b4b9ce06 admin admin] PATH_INFO: `/v3/projects` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:47.121301 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca4e6f3-ab40-4acd-865f-e307b4b9ce06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:47.125823 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca4e6f3-ab40-4acd-865f-e307b4b9ce06 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:47.151904 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76205879-91b5-43de-bdf7-b4f8678aef4c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:47.153533 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76205879-91b5-43de-bdf7-b4f8678aef4c service nova] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:47.189177 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 840/1688] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:41:46 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 589 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 08:41:47.191429 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5a1d14-e8ba-4132-a0a1-a4cbecbf7586 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:47.191724 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5a1d14-e8ba-4132-a0a1-a4cbecbf7586 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:47.192044 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5a1d14-e8ba-4132-a0a1-a4cbecbf7586 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:47.193487 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 841/1689] 10.222.0.57 () {54 vars in 912 bytes} [Wed Aug 10 08:41:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:41:47.199784 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa306194-301f-4957-b0a0-361b5f35228c None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:47.200076 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa306194-301f-4957-b0a0-361b5f35228c None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:47.200394 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa306194-301f-4957-b0a0-361b5f35228c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:47.244083 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 849/1690] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 08:41:46 2022] POST /identity/v3/projects => generated 381 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 08:41:47.249527 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca4e6f3-ab40-4acd-865f-e307b4b9ce06 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:47.338642 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-fa306194-301f-4957-b0a0-361b5f35228c None None] MFA Rules not processed for user `d4863ecff21e408c9deca645b88d3727`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:41:47.340794 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-fa306194-301f-4957-b0a0-361b5f35228c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:41:47.428177 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:41:47.460982 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca4e6f3-ab40-4acd-865f-e307b4b9ce06 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:47.480678 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ca86b9-abe7-4874-940d-508a3faf3f0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:47.481566 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ca86b9-abe7-4874-940d-508a3faf3f0d admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:47.482778 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ca86b9-abe7-4874-940d-508a3faf3f0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:47.484161 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ca86b9-abe7-4874-940d-508a3faf3f0d admin admin] PATH_INFO: `/v3/users` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:47.485332 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ca86b9-abe7-4874-940d-508a3faf3f0d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:47.488293 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ca86b9-abe7-4874-940d-508a3faf3f0d admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:47.545417 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 842/1691] 10.222.0.57 () {58 vars in 1009 bytes} [Wed Aug 10 08:41:47 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 346 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 08:41:47.595521 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 850/1692] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 08:41:47 2022] POST /identity/v3/users => generated 297 bytes in 347 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:41:47.600263 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa306194-301f-4957-b0a0-361b5f35228c None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:47.621903 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa306194-301f-4957-b0a0-361b5f35228c None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:47.634551 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00615b6-984b-4f86-b931-7a03b3eee81a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:47.636681 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00615b6-984b-4f86-b931-7a03b3eee81a admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:47.637003 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00615b6-984b-4f86-b931-7a03b3eee81a admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:47.637327 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00615b6-984b-4f86-b931-7a03b3eee81a admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:47.638193 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00615b6-984b-4f86-b931-7a03b3eee81a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:47.651352 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00615b6-984b-4f86-b931-7a03b3eee81a admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:47.669756 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ca86b9-abe7-4874-940d-508a3faf3f0d admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:47.678063 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 843/1693] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:41:47 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:41:47.683763 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00615b6-984b-4f86-b931-7a03b3eee81a admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:47.684691 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ca86b9-abe7-4874-940d-508a3faf3f0d admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:47.687395 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-b00615b6-984b-4f86-b931-7a03b3eee81a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:41:47.697630 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c1928d-f653-414d-a515-20a265d2f4cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:47.698508 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c1928d-f653-414d-a515-20a265d2f4cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:47.700977 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c1928d-f653-414d-a515-20a265d2f4cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:47.701387 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c1928d-f653-414d-a515-20a265d2f4cf admin admin] PATH_INFO: `/v3/projects/50f38633e8454957ab1867de4ceab7ab/users/3998061ded274c558f9c38a234a1f184/roles/ec3751692d9c4ecf81b4a71c936c46be` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:47.719620 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00615b6-984b-4f86-b931-7a03b3eee81a admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:47.724834 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7243120-13b0-489c-865a-86b10039ca73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:47.726257 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7243120-13b0-489c-865a-86b10039ca73 service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:47.726542 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7243120-13b0-489c-865a-86b10039ca73 service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:47.726815 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7243120-13b0-489c-865a-86b10039ca73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:47.736051 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c1928d-f653-414d-a515-20a265d2f4cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=50f38633e8454957ab1867de4ceab7ab, user_id=3998061ded274c558f9c38a234a1f184, role_id=ec3751692d9c4ecf81b4a71c936c46be)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:47.738184 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7243120-13b0-489c-865a-86b10039ca73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:47.739821 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7243120-13b0-489c-865a-86b10039ca73 service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:47.740219 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c1928d-f653-414d-a515-20a265d2f4cf admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:47.763697 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 844/1694] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:41:47 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:41:47.815302 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 851/1695] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 08:41:47 2022] PUT /identity/v3/projects/50f38633e8454957ab1867de4ceab7ab/users/3998061ded274c558f9c38a234a1f184/roles/ec3751692d9c4ecf81b4a71c936c46be => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:41:47.817417 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7243120-13b0-489c-865a-86b10039ca73 service neutron] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:47.838097 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7243120-13b0-489c-865a-86b10039ca73 service neutron] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:47.895969 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:41:47.937846 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70f2a5a-d8cf-43ca-985f-2ee2db18714c None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:47.938454 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70f2a5a-d8cf-43ca-985f-2ee2db18714c None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:47.939006 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70f2a5a-d8cf-43ca-985f-2ee2db18714c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:47.985028 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-c70f2a5a-d8cf-43ca-985f-2ee2db18714c None None] MFA Rules not processed for user `258ec0f975eb47738f4dac356c0578e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:41:47.988311 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-c70f2a5a-d8cf-43ca-985f-2ee2db18714c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:41:48.025962 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db93f5d1-1125-40b0-9563-180ffd527cf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:48.027908 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db93f5d1-1125-40b0-9563-180ffd527cf0 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:48.028534 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db93f5d1-1125-40b0-9563-180ffd527cf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:48.028534 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db93f5d1-1125-40b0-9563-180ffd527cf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:48.029150 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db93f5d1-1125-40b0-9563-180ffd527cf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:48.040975 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db93f5d1-1125-40b0-9563-180ffd527cf0 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:48.074472 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 845/1696] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:41:47 2022] GET /identity/v3/roles => generated 1420 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:41:48.079171 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db93f5d1-1125-40b0-9563-180ffd527cf0 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:48.104533 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db93f5d1-1125-40b0-9563-180ffd527cf0 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:48.111510 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5c6695-dc3a-41e5-885f-e81ac72216f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:48.111510 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5c6695-dc3a-41e5-885f-e81ac72216f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:48.111510 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5c6695-dc3a-41e5-885f-e81ac72216f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:48.111891 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5c6695-dc3a-41e5-885f-e81ac72216f9 admin admin] PATH_INFO: `/v3/projects/50f38633e8454957ab1867de4ceab7ab/users/3998061ded274c558f9c38a234a1f184/roles/ec3751692d9c4ecf81b4a71c936c46be` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:48.114008 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5c6695-dc3a-41e5-885f-e81ac72216f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50f38633e8454957ab1867de4ceab7ab, user_id=3998061ded274c558f9c38a234a1f184, role_id=ec3751692d9c4ecf81b4a71c936c46be)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:48.116944 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5c6695-dc3a-41e5-885f-e81ac72216f9 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:48.137223 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 846/1697] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 08:41:48 2022] PUT /identity/v3/projects/50f38633e8454957ab1867de4ceab7ab/users/3998061ded274c558f9c38a234a1f184/roles/ec3751692d9c4ecf81b4a71c936c46be => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:41:48.150460 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c292dc1-d43d-4d3f-aedd-4917425036be None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:48.150977 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 852/1698] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:41:47 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 08:41:48.151692 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c292dc1-d43d-4d3f-aedd-4917425036be None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:48.152167 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c292dc1-d43d-4d3f-aedd-4917425036be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:48.221500 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-6c292dc1-d43d-4d3f-aedd-4917425036be None None] MFA Rules not processed for user `3998061ded274c558f9c38a234a1f184`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:41:48.223679 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-6c292dc1-d43d-4d3f-aedd-4917425036be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:41:48.339273 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 847/1699] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:41:48 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:41:48.356409 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70f2a5a-d8cf-43ca-985f-2ee2db18714c None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:48.356409 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-c70f2a5a-d8cf-43ca-985f-2ee2db18714c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:41:48.361847 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c292dc1-d43d-4d3f-aedd-4917425036be None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:48.366617 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-6c292dc1-d43d-4d3f-aedd-4917425036be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:41:48.505232 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70f2a5a-d8cf-43ca-985f-2ee2db18714c None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:48.551031 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c292dc1-d43d-4d3f-aedd-4917425036be None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:48.563375 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9c2a3a-aa0d-415d-a40f-1dfaef989653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:48.566777 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9c2a3a-aa0d-415d-a40f-1dfaef989653 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:48.567144 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9c2a3a-aa0d-415d-a40f-1dfaef989653 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:48.567523 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9c2a3a-aa0d-415d-a40f-1dfaef989653 admin admin] PATH_INFO: `/v3/projects/6f9f6bd4786942b8a5297dd04d0c79ba` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:48.574429 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9c2a3a-aa0d-415d-a40f-1dfaef989653 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f9f6bd4786942b8a5297dd04d0c79ba)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:48.577715 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9c2a3a-aa0d-415d-a40f-1dfaef989653 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:48.615981 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f1aba0-2dcc-44f7-92c3-35736c162e12 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ba74e28ced4cbba857f54222dfbe41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:48.617026 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f1aba0-2dcc-44f7-92c3-35736c162e12 service cinder] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:48.617447 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f1aba0-2dcc-44f7-92c3-35736c162e12 service cinder] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:48.617903 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f1aba0-2dcc-44f7-92c3-35736c162e12 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:48.633846 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f1aba0-2dcc-44f7-92c3-35736c162e12 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:48.635432 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f1aba0-2dcc-44f7-92c3-35736c162e12 service cinder] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:48.649486 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 853/1700] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:41:48 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:41:48.650948 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870275bb-5f4b-491d-9054-91023c3428b1 None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:48.651230 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870275bb-5f4b-491d-9054-91023c3428b1 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:48.651603 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870275bb-5f4b-491d-9054-91023c3428b1 None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:48.653505 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 854/1701] 10.222.0.57 () {54 vars in 912 bytes} [Wed Aug 10 08:41:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:41:48.669067 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe855b-7e49-4450-8f72-289f26921434 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:48.669272 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe855b-7e49-4450-8f72-289f26921434 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:48.669470 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe855b-7e49-4450-8f72-289f26921434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:48.710147 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-4afe855b-7e49-4450-8f72-289f26921434 None None] MFA Rules not processed for user `d4863ecff21e408c9deca645b88d3727`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:41:48.711502 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-4afe855b-7e49-4450-8f72-289f26921434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:41:48.726529 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:41:48.744004 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-bb9c2a3a-aa0d-415d-a40f-1dfaef989653 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f9f6bd4786942b8a5297dd04d0c79ba', 'request_id': 'req-bb9c2a3a-aa0d-415d-a40f-1dfaef989653'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:41:48.744271 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:41:48.809809 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 855/1702] 10.222.0.57 () {58 vars in 1009 bytes} [Wed Aug 10 08:41:48 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:41:48.816466 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afe855b-7e49-4450-8f72-289f26921434 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:48.831980 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afe855b-7e49-4450-8f72-289f26921434 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:48.869872 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-bb9c2a3a-aa0d-415d-a40f-1dfaef989653 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f9f6bd4786942b8a5297dd04d0c79ba', 'request_id': 'req-bb9c2a3a-aa0d-415d-a40f-1dfaef989653'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:41:48.888391 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-bb9c2a3a-aa0d-415d-a40f-1dfaef989653 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f9f6bd4786942b8a5297dd04d0c79ba', 'request_id': 'req-bb9c2a3a-aa0d-415d-a40f-1dfaef989653'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:41:48.929032 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-bb9c2a3a-aa0d-415d-a40f-1dfaef989653 admin admin] The token cache is being invalidate because project 6f9f6bd4786942b8a5297dd04d0c79ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 08:41:48.929331 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-bb9c2a3a-aa0d-415d-a40f-1dfaef989653 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:41:48.930065 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-bb9c2a3a-aa0d-415d-a40f-1dfaef989653 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:41:48.937045 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 848/1703] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 08:41:48 2022] DELETE /identity/v3/projects/6f9f6bd4786942b8a5297dd04d0c79ba => generated 0 bytes in 575 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:41:48.937045 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2b6f6e-b957-4b3e-af13-67ccad2e6cf8 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:48.937045 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2b6f6e-b957-4b3e-af13-67ccad2e6cf8 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:48.937045 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2b6f6e-b957-4b3e-af13-67ccad2e6cf8 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:48.938018 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 849/1704] 10.222.0.57 () {54 vars in 961 bytes} [Wed Aug 10 08:41:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:41:48.939135 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb55063-a262-49df-8310-4efb51dc8939 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:48.939939 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb55063-a262-49df-8310-4efb51dc8939 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:48.940518 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb55063-a262-49df-8310-4efb51dc8939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:48.943736 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49442a7-14fb-4eed-8158-82070603d78b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:48.944519 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49442a7-14fb-4eed-8158-82070603d78b service neutron] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:48.944880 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49442a7-14fb-4eed-8158-82070603d78b service neutron] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:48.945413 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49442a7-14fb-4eed-8158-82070603d78b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:48.947694 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-e49442a7-14fb-4eed-8158-82070603d78b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:41:48.960702 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49442a7-14fb-4eed-8158-82070603d78b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:48.962631 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49442a7-14fb-4eed-8158-82070603d78b service neutron] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:48.975145 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-abb55063-a262-49df-8310-4efb51dc8939 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:41:48.975775 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 856/1705] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:41:48 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 08:41:48.976605 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-abb55063-a262-49df-8310-4efb51dc8939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:41:48.977574 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cbb7fe-d608-4fe0-abfa-1b6b5da46da2 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:48.977926 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cbb7fe-d608-4fe0-abfa-1b6b5da46da2 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:48.978206 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cbb7fe-d608-4fe0-abfa-1b6b5da46da2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:49.045003 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-29cbb7fe-d608-4fe0-abfa-1b6b5da46da2 None None] MFA Rules not processed for user `2912307c56594125b157f1cfdf7c96a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:41:49.055108 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-29cbb7fe-d608-4fe0-abfa-1b6b5da46da2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:41:49.094226 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 850/1706] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:41:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:41:49.094226 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb55063-a262-49df-8310-4efb51dc8939 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:49.094226 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-abb55063-a262-49df-8310-4efb51dc8939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:41:49.179481 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 857/1707] 10.222.0.57 () {58 vars in 1058 bytes} [Wed Aug 10 08:41:48 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:41:49.181481 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be0223a-191c-4fcc-9762-416e7a76258d None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:49.182119 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be0223a-191c-4fcc-9762-416e7a76258d None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:49.182604 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be0223a-191c-4fcc-9762-416e7a76258d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:49.204764 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-2be0223a-191c-4fcc-9762-416e7a76258d None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:41:49.207669 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-2be0223a-191c-4fcc-9762-416e7a76258d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:41:49.233689 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb55063-a262-49df-8310-4efb51dc8939 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:49.239475 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 858/1708] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:41:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:41:49.240417 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be0223a-191c-4fcc-9762-416e7a76258d None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:49.252778 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ace35b-c14a-4b0f-8d50-c62107161f61 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ba74e28ced4cbba857f54222dfbe41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:49.254590 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ace35b-c14a-4b0f-8d50-c62107161f61 service cinder] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:49.257713 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ace35b-c14a-4b0f-8d50-c62107161f61 service cinder] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:49.257713 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ace35b-c14a-4b0f-8d50-c62107161f61 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:49.258100 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-d1ace35b-c14a-4b0f-8d50-c62107161f61 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:41:49.271477 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be0223a-191c-4fcc-9762-416e7a76258d None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:49.277330 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12c79e7-e5e1-4a9c-83c9-75b419dc508c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2912307c56594125b157f1cfdf7c96a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:49.278931 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12c79e7-e5e1-4a9c-83c9-75b419dc508c service placement] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:49.280660 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12c79e7-e5e1-4a9c-83c9-75b419dc508c service placement] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:49.280660 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12c79e7-e5e1-4a9c-83c9-75b419dc508c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:49.284064 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-a12c79e7-e5e1-4a9c-83c9-75b419dc508c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:41:49.298487 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12c79e7-e5e1-4a9c-83c9-75b419dc508c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:49.301507 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12c79e7-e5e1-4a9c-83c9-75b419dc508c service placement] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:49.315317 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 859/1709] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:41:49 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:41:49.315690 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12c79e7-e5e1-4a9c-83c9-75b419dc508c service placement] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:49.333543 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12c79e7-e5e1-4a9c-83c9-75b419dc508c service placement] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:49.345945 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d86ad0-d213-48ab-8c4b-7867e8d8f604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:49.347347 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d86ad0-d213-48ab-8c4b-7867e8d8f604 admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:49.348661 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d86ad0-d213-48ab-8c4b-7867e8d8f604 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:49.349087 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d86ad0-d213-48ab-8c4b-7867e8d8f604 admin admin] PATH_INFO: `/v3/domains` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:49.351085 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d86ad0-d213-48ab-8c4b-7867e8d8f604 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:41:49.352398 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d86ad0-d213-48ab-8c4b-7867e8d8f604 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:49.355584 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d86ad0-d213-48ab-8c4b-7867e8d8f604 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:49.367114 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ace35b-c14a-4b0f-8d50-c62107161f61 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:49.369181 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ace35b-c14a-4b0f-8d50-c62107161f61 service cinder] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:49.423653 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 860/1710] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Aug 10 08:41:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:41:49.424756 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 851/1711] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:41:49 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:41:49.429068 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d86ad0-d213-48ab-8c4b-7867e8d8f604 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:49.474631 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d86ad0-d213-48ab-8c4b-7867e8d8f604 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:49.500152 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51dab4b-f4d9-423d-bf9e-9b5c5c6bd4d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:49.500152 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51dab4b-f4d9-423d-bf9e-9b5c5c6bd4d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:49.500152 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51dab4b-f4d9-423d-bf9e-9b5c5c6bd4d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:49.500152 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51dab4b-f4d9-423d-bf9e-9b5c5c6bd4d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:49.500580 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51dab4b-f4d9-423d-bf9e-9b5c5c6bd4d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:49.518832 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51dab4b-f4d9-423d-bf9e-9b5c5c6bd4d8 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:49.695979 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 861/1712] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 08:41:49 2022] POST /identity/v3/projects => generated 375 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:41:49.697139 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ace35b-c14a-4b0f-8d50-c62107161f61 service cinder] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:49.713637 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ace35b-c14a-4b0f-8d50-c62107161f61 service cinder] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:49.741350 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:41:49.833240 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce93016-c601-4c7d-a179-b953fd4e39ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:49.835332 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce93016-c601-4c7d-a179-b953fd4e39ba admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:49.835898 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce93016-c601-4c7d-a179-b953fd4e39ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:49.836120 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce93016-c601-4c7d-a179-b953fd4e39ba admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:49.836963 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce93016-c601-4c7d-a179-b953fd4e39ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:49.839829 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce93016-c601-4c7d-a179-b953fd4e39ba admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:49.982386 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 852/1713] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 08:41:49 2022] POST /identity/v3/users => generated 288 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 08:41:49.989072 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51dab4b-f4d9-423d-bf9e-9b5c5c6bd4d8 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:50.005708 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51dab4b-f4d9-423d-bf9e-9b5c5c6bd4d8 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:50.012177 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0190524f-2f49-4ef3-802b-1981c02377f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:50.013016 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0190524f-2f49-4ef3-802b-1981c02377f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:50.013823 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0190524f-2f49-4ef3-802b-1981c02377f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:50.014293 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0190524f-2f49-4ef3-802b-1981c02377f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:50.015175 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0190524f-2f49-4ef3-802b-1981c02377f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:50.018136 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0190524f-2f49-4ef3-802b-1981c02377f7 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:50.033072 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:41:50.074794 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 862/1714] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:41:49 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:41:50.075075 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce93016-c601-4c7d-a179-b953fd4e39ba admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:50.088468 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce93016-c601-4c7d-a179-b953fd4e39ba admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:50.094847 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1a848d-4507-412a-abfb-8b5f4f180832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:50.095823 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1a848d-4507-412a-abfb-8b5f4f180832 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:50.096287 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1a848d-4507-412a-abfb-8b5f4f180832 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:50.096792 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1a848d-4507-412a-abfb-8b5f4f180832 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:50.097874 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1a848d-4507-412a-abfb-8b5f4f180832 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:50.099661 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1a848d-4507-412a-abfb-8b5f4f180832 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:50.116742 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 853/1715] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:41:50 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:41:50.121928 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0190524f-2f49-4ef3-802b-1981c02377f7 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:50.161981 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0190524f-2f49-4ef3-802b-1981c02377f7 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:50.178020 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9ec419-b1d6-44ec-ab58-48d19b28f017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:50.179222 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9ec419-b1d6-44ec-ab58-48d19b28f017 admin admin] REQUEST_METHOD: `PUT` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:50.179574 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9ec419-b1d6-44ec-ab58-48d19b28f017 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:50.180885 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9ec419-b1d6-44ec-ab58-48d19b28f017 admin admin] PATH_INFO: `/v3/projects/5345961c2f544ff78cc4832cdf1587bc/users/e1c233994efe4b31b9ba493f7f598d27/roles/44b64db484f24a46bf0b7ced46b519bf` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:50.205523 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9ec419-b1d6-44ec-ab58-48d19b28f017 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5345961c2f544ff78cc4832cdf1587bc, user_id=e1c233994efe4b31b9ba493f7f598d27, role_id=44b64db484f24a46bf0b7ced46b519bf)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:50.208430 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9ec419-b1d6-44ec-ab58-48d19b28f017 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:50.288336 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 863/1716] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 08:41:50 2022] PUT /identity/v3/projects/5345961c2f544ff78cc4832cdf1587bc/users/e1c233994efe4b31b9ba493f7f598d27/roles/44b64db484f24a46bf0b7ced46b519bf => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:41:50.303186 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c22d62-f931-421f-9afb-1e5bdeb4e041 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:50.304314 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c22d62-f931-421f-9afb-1e5bdeb4e041 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:50.304729 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c22d62-f931-421f-9afb-1e5bdeb4e041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:50.456383 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-b3c22d62-f931-421f-9afb-1e5bdeb4e041 None None] MFA Rules not processed for user `e1c233994efe4b31b9ba493f7f598d27`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:41:50.463019 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-b3c22d62-f931-421f-9afb-1e5bdeb4e041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:41:50.589363 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 854/1717] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:41:50 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:41:51.168008 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a945db7d-0033-4359-b033-98c0e30366ff None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:51.168559 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a945db7d-0033-4359-b033-98c0e30366ff None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:51.169058 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a945db7d-0033-4359-b033-98c0e30366ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:51.196862 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-a945db7d-0033-4359-b033-98c0e30366ff None None] MFA Rules not processed for user `9797df3e05944552b741f53773abf80a`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:41:51.199674 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-a945db7d-0033-4359-b033-98c0e30366ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:41:51.337683 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c22d62-f931-421f-9afb-1e5bdeb4e041 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:51.339370 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-b3c22d62-f931-421f-9afb-1e5bdeb4e041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:41:51.379054 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 864/1718] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:41:51 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:41:51.655243 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c22d62-f931-421f-9afb-1e5bdeb4e041 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:51.664294 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60623a45-a79c-4e64-8ea2-21c82707b06f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a943008193914f76a663d489bd73e873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:51.665622 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60623a45-a79c-4e64-8ea2-21c82707b06f service nova] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:51.665908 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60623a45-a79c-4e64-8ea2-21c82707b06f service nova] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:51.666306 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60623a45-a79c-4e64-8ea2-21c82707b06f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:51.683857 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60623a45-a79c-4e64-8ea2-21c82707b06f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:51.686465 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60623a45-a79c-4e64-8ea2-21c82707b06f service nova] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:51.716050 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 855/1719] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:41:51 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:41:51.995377 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0404e528-6b2e-47f9-80e4-c8e00a563b43 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:51.997772 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0404e528-6b2e-47f9-80e4-c8e00a563b43 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:51.999570 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0404e528-6b2e-47f9-80e4-c8e00a563b43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:52.082442 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-0404e528-6b2e-47f9-80e4-c8e00a563b43 None None] MFA Rules not processed for user `4ec7ba61e65447a5a87b0f8fcc2b946c`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:41:52.092744 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-0404e528-6b2e-47f9-80e4-c8e00a563b43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:41:52.351057 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 865/1720] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:41:51 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 360 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:41:52.543499 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fa0d3f-d743-4f42-9738-619b126dcf9f None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:52.544036 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fa0d3f-d743-4f42-9738-619b126dcf9f None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:52.544036 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fa0d3f-d743-4f42-9738-619b126dcf9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:52.567307 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:41:52.590976 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-28fa0d3f-d743-4f42-9738-619b126dcf9f None None] MFA Rules not processed for user `3bf116fade674838b9219e15dd20eda2`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:41:52.592243 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-28fa0d3f-d743-4f42-9738-619b126dcf9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:41:52.708401 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 856/1721] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:41:52 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:41:55.151327 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4edc7b8-a0cf-424b-a0ff-a9b75fb02e3a None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:55.151327 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4edc7b8-a0cf-424b-a0ff-a9b75fb02e3a None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:55.151327 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4edc7b8-a0cf-424b-a0ff-a9b75fb02e3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:55.200669 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fa0d3f-d743-4f42-9738-619b126dcf9f None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:55.338596 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-a4edc7b8-a0cf-424b-a0ff-a9b75fb02e3a None None] MFA Rules not processed for user `c140d31bd2b84c48a170e3ca90a041ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:41:55.360711 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-a4edc7b8-a0cf-424b-a0ff-a9b75fb02e3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:41:55.472780 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fa0d3f-d743-4f42-9738-619b126dcf9f None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:55.488423 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb68d947-1f74-4f76-8091-12ce06058c44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:55.489172 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb68d947-1f74-4f76-8091-12ce06058c44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:55.489448 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb68d947-1f74-4f76-8091-12ce06058c44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:55.489715 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb68d947-1f74-4f76-8091-12ce06058c44 admin admin] PATH_INFO: `/v3/users/e1c233994efe4b31b9ba493f7f598d27` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:55.491609 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb68d947-1f74-4f76-8091-12ce06058c44 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1c233994efe4b31b9ba493f7f598d27)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:55.514169 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb68d947-1f74-4f76-8091-12ce06058c44 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:55.584332 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:41:55.620880 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 866/1722] 10.222.0.57 () {58 vars in 1009 bytes} [Wed Aug 10 08:41:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 474 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:41:55.903475 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-eb68d947-1f74-4f76-8091-12ce06058c44 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1c233994efe4b31b9ba493f7f598d27'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:41:55.940446 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-eb68d947-1f74-4f76-8091-12ce06058c44 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1c233994efe4b31b9ba493f7f598d27'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:41:55.965495 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-eb68d947-1f74-4f76-8091-12ce06058c44 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1c233994efe4b31b9ba493f7f598d27'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:41:55.967070 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-eb68d947-1f74-4f76-8091-12ce06058c44 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1c233994efe4b31b9ba493f7f598d27'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:41:55.969735 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-eb68d947-1f74-4f76-8091-12ce06058c44 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1c233994efe4b31b9ba493f7f598d27'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:41:56.117518 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4edc7b8-a0cf-424b-a0ff-a9b75fb02e3a None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:56.119306 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-a4edc7b8-a0cf-424b-a0ff-a9b75fb02e3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:41:56.135885 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 857/1723] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 08:41:55 2022] DELETE /identity/v3/users/e1c233994efe4b31b9ba493f7f598d27 => generated 0 bytes in 937 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:41:56.140853 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773076a8-1378-4782-a00f-59e72b70247b None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:56.141190 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773076a8-1378-4782-a00f-59e72b70247b None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:56.141470 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773076a8-1378-4782-a00f-59e72b70247b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:56.222550 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-773076a8-1378-4782-a00f-59e72b70247b None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:41:56.230201 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-773076a8-1378-4782-a00f-59e72b70247b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:41:56.321765 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:41:56.370778 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4edc7b8-a0cf-424b-a0ff-a9b75fb02e3a None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:56.418994 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 858/1724] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:41:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:41:56.461219 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773076a8-1378-4782-a00f-59e72b70247b None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:56.463426 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-773076a8-1378-4782-a00f-59e72b70247b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:41:56.539995 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc418724-969b-40ff-92e0-f3c2652a93d2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4b26ba3a9e04c38b52db2a46198a53c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:56.541023 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc418724-969b-40ff-92e0-f3c2652a93d2 service swift] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:56.541454 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc418724-969b-40ff-92e0-f3c2652a93d2 service swift] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:56.541903 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc418724-969b-40ff-92e0-f3c2652a93d2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:56.546688 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-cc418724-969b-40ff-92e0-f3c2652a93d2 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:41:56.654407 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773076a8-1378-4782-a00f-59e72b70247b None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:56.663327 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01186b82-26d8-4a1f-9b88-b67bf1381008 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:56.665131 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01186b82-26d8-4a1f-9b88-b67bf1381008 service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:56.665520 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01186b82-26d8-4a1f-9b88-b67bf1381008 service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:56.665904 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01186b82-26d8-4a1f-9b88-b67bf1381008 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:56.682227 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01186b82-26d8-4a1f-9b88-b67bf1381008 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:56.683800 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01186b82-26d8-4a1f-9b88-b67bf1381008 service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:56.712485 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 859/1725] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:41:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:41:56.714619 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc418724-969b-40ff-92e0-f3c2652a93d2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:56.716534 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc418724-969b-40ff-92e0-f3c2652a93d2 service swift] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:56.763139 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 867/1726] 10.222.0.57 () {60 vars in 1384 bytes} [Wed Aug 10 08:41:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 647 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 08:41:57.476705 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01186b82-26d8-4a1f-9b88-b67bf1381008 service neutron] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:57.478166 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-01186b82-26d8-4a1f-9b88-b67bf1381008 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:41:57.511981 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01186b82-26d8-4a1f-9b88-b67bf1381008 service neutron] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:57.517887 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cc8c54-de98-417a-bebf-6e6b6d1a598f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:57.518620 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cc8c54-de98-417a-bebf-6e6b6d1a598f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:57.518960 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cc8c54-de98-417a-bebf-6e6b6d1a598f admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:57.519307 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cc8c54-de98-417a-bebf-6e6b6d1a598f admin admin] PATH_INFO: `/v3/projects/5345961c2f544ff78cc4832cdf1587bc` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:57.520657 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cc8c54-de98-417a-bebf-6e6b6d1a598f admin admin] RBAC: Authorizing `identity:delete_project(project_id=5345961c2f544ff78cc4832cdf1587bc)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:57.522718 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cc8c54-de98-417a-bebf-6e6b6d1a598f admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:57.645272 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-61cc8c54-de98-417a-bebf-6e6b6d1a598f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5345961c2f544ff78cc4832cdf1587bc', 'request_id': 'req-61cc8c54-de98-417a-bebf-6e6b6d1a598f'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:41:57.707832 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-61cc8c54-de98-417a-bebf-6e6b6d1a598f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5345961c2f544ff78cc4832cdf1587bc', 'request_id': 'req-61cc8c54-de98-417a-bebf-6e6b6d1a598f'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:41:57.717906 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-61cc8c54-de98-417a-bebf-6e6b6d1a598f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5345961c2f544ff78cc4832cdf1587bc', 'request_id': 'req-61cc8c54-de98-417a-bebf-6e6b6d1a598f'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:41:57.734297 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-61cc8c54-de98-417a-bebf-6e6b6d1a598f admin admin] The token cache is being invalidate because project 5345961c2f544ff78cc4832cdf1587bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 08:41:57.734926 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-61cc8c54-de98-417a-bebf-6e6b6d1a598f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:41:57.737062 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-61cc8c54-de98-417a-bebf-6e6b6d1a598f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:41:57.744728 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 860/1727] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 08:41:57 2022] DELETE /identity/v3/projects/5345961c2f544ff78cc4832cdf1587bc => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:41:57.748711 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e937f70b-7a7c-49e7-8ac6-a2019ae30bd8 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:57.749236 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e937f70b-7a7c-49e7-8ac6-a2019ae30bd8 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:57.749890 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e937f70b-7a7c-49e7-8ac6-a2019ae30bd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:57.783361 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-e937f70b-7a7c-49e7-8ac6-a2019ae30bd8 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:41:57.784899 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-e937f70b-7a7c-49e7-8ac6-a2019ae30bd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:41:57.812156 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:41:57.864099 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 868/1728] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:41:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:41:57.872038 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2350f027-960b-429d-a2ee-925d8eb02220 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:57.872386 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2350f027-960b-429d-a2ee-925d8eb02220 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:57.872694 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2350f027-960b-429d-a2ee-925d8eb02220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:57.895996 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-2350f027-960b-429d-a2ee-925d8eb02220 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:41:57.897928 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-2350f027-960b-429d-a2ee-925d8eb02220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:41:57.909007 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 861/1729] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:41:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 08:41:57.912987 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e937f70b-7a7c-49e7-8ac6-a2019ae30bd8 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:57.927004 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e937f70b-7a7c-49e7-8ac6-a2019ae30bd8 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:57.930404 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416a18dc-a236-4c9f-b5c6-d793d60ea1fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:57.931025 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416a18dc-a236-4c9f-b5c6-d793d60ea1fe admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:57.931408 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416a18dc-a236-4c9f-b5c6-d793d60ea1fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:57.931554 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416a18dc-a236-4c9f-b5c6-d793d60ea1fe admin admin] PATH_INFO: `/v3/domains` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:57.932234 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416a18dc-a236-4c9f-b5c6-d793d60ea1fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:41:57.932532 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416a18dc-a236-4c9f-b5c6-d793d60ea1fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:57.934150 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416a18dc-a236-4c9f-b5c6-d793d60ea1fe admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:57.948635 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 869/1730] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Aug 10 08:41:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:41:57.957482 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2350f027-960b-429d-a2ee-925d8eb02220 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:57.972780 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2350f027-960b-429d-a2ee-925d8eb02220 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:57.980623 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f295d77d-9aba-4824-b62f-2d7c45b0b558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:57.981537 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f295d77d-9aba-4824-b62f-2d7c45b0b558 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:57.981967 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f295d77d-9aba-4824-b62f-2d7c45b0b558 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:57.982413 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f295d77d-9aba-4824-b62f-2d7c45b0b558 admin admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:57.983243 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f295d77d-9aba-4824-b62f-2d7c45b0b558 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:57.986519 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f295d77d-9aba-4824-b62f-2d7c45b0b558 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:58.099245 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 862/1731] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 08:41:57 2022] POST /identity/v3/projects => generated 365 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 08:41:58.099984 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416a18dc-a236-4c9f-b5c6-d793d60ea1fe admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:58.112691 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416a18dc-a236-4c9f-b5c6-d793d60ea1fe admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:58.195328 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd7b469-6a9d-43e4-8129-b342df3a814c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:58.196207 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd7b469-6a9d-43e4-8129-b342df3a814c admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:58.196671 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd7b469-6a9d-43e4-8129-b342df3a814c admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:58.197167 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd7b469-6a9d-43e4-8129-b342df3a814c admin admin] PATH_INFO: `/v3/users` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:58.198979 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd7b469-6a9d-43e4-8129-b342df3a814c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:58.203500 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd7b469-6a9d-43e4-8129-b342df3a814c admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:58.298765 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 870/1732] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 08:41:58 2022] POST /identity/v3/users => generated 283 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 08:41:58.304776 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f295d77d-9aba-4824-b62f-2d7c45b0b558 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:58.313735 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f295d77d-9aba-4824-b62f-2d7c45b0b558 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:58.333577 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6ad5b6-3f5c-46fc-8888-5f53aadbb370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:58.333577 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6ad5b6-3f5c-46fc-8888-5f53aadbb370 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:58.333577 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6ad5b6-3f5c-46fc-8888-5f53aadbb370 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:58.333577 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6ad5b6-3f5c-46fc-8888-5f53aadbb370 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:58.333989 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6ad5b6-3f5c-46fc-8888-5f53aadbb370 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:58.333989 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6ad5b6-3f5c-46fc-8888-5f53aadbb370 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:58.350386 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 863/1733] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:41:58 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:41:58.354502 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd7b469-6a9d-43e4-8129-b342df3a814c admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:58.366063 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd7b469-6a9d-43e4-8129-b342df3a814c admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:58.371023 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd5b5c8-2c23-4f3f-872b-aa0d2a210de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:58.371664 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd5b5c8-2c23-4f3f-872b-aa0d2a210de5 admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:58.371946 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd5b5c8-2c23-4f3f-872b-aa0d2a210de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:58.372219 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd5b5c8-2c23-4f3f-872b-aa0d2a210de5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:58.373809 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd5b5c8-2c23-4f3f-872b-aa0d2a210de5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:58.375408 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd5b5c8-2c23-4f3f-872b-aa0d2a210de5 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:58.395659 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 871/1734] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:41:58 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:41:58.399609 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6ad5b6-3f5c-46fc-8888-5f53aadbb370 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:58.416129 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6ad5b6-3f5c-46fc-8888-5f53aadbb370 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:58.421292 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31d8014-094f-4ff7-b971-95ef39fc40f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:58.422063 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31d8014-094f-4ff7-b971-95ef39fc40f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:58.422669 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31d8014-094f-4ff7-b971-95ef39fc40f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:58.423022 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31d8014-094f-4ff7-b971-95ef39fc40f1 admin admin] PATH_INFO: `/v3/projects/17abac2aed18465c85d6ad6b8bd2d7f7/users/600bbed27a5344f9ad14ffc91ea2b1d7/roles/44b64db484f24a46bf0b7ced46b519bf` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:58.452181 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31d8014-094f-4ff7-b971-95ef39fc40f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17abac2aed18465c85d6ad6b8bd2d7f7, user_id=600bbed27a5344f9ad14ffc91ea2b1d7, role_id=44b64db484f24a46bf0b7ced46b519bf)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:58.455604 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31d8014-094f-4ff7-b971-95ef39fc40f1 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:58.460573 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:41:58.541031 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 864/1735] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 08:41:58 2022] PUT /identity/v3/projects/17abac2aed18465c85d6ad6b8bd2d7f7/users/600bbed27a5344f9ad14ffc91ea2b1d7/roles/44b64db484f24a46bf0b7ced46b519bf => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:41:58.545147 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f379901f-e083-40d5-ad29-2ffbb43ac8bc None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:58.545577 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f379901f-e083-40d5-ad29-2ffbb43ac8bc None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:58.546145 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f379901f-e083-40d5-ad29-2ffbb43ac8bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:58.590020 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-f379901f-e083-40d5-ad29-2ffbb43ac8bc None None] MFA Rules not processed for user `600bbed27a5344f9ad14ffc91ea2b1d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:41:58.595361 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-f379901f-e083-40d5-ad29-2ffbb43ac8bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:41:58.699916 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 872/1736] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:41:58 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:41:58.706448 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31d8014-094f-4ff7-b971-95ef39fc40f1 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:58.797595 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31d8014-094f-4ff7-b971-95ef39fc40f1 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:58.804244 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc5fd57-d0ed-44b7-aee4-37a8b1b48407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:58.804892 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc5fd57-d0ed-44b7-aee4-37a8b1b48407 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:58.805202 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc5fd57-d0ed-44b7-aee4-37a8b1b48407 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:58.805529 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc5fd57-d0ed-44b7-aee4-37a8b1b48407 admin admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:58.806174 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc5fd57-d0ed-44b7-aee4-37a8b1b48407 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:58.808143 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc5fd57-d0ed-44b7-aee4-37a8b1b48407 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:58.902104 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 865/1737] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 08:41:58 2022] POST /identity/v3/projects => generated 361 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:41:58.906055 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f379901f-e083-40d5-ad29-2ffbb43ac8bc None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:58.918925 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f379901f-e083-40d5-ad29-2ffbb43ac8bc None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:58.932015 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:41:58.977455 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839afc80-ff04-4a61-a09d-7a92e929bdde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:58.978067 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839afc80-ff04-4a61-a09d-7a92e929bdde admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:58.978339 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839afc80-ff04-4a61-a09d-7a92e929bdde admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:58.978604 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839afc80-ff04-4a61-a09d-7a92e929bdde admin admin] PATH_INFO: `/v3/users` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:58.979214 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839afc80-ff04-4a61-a09d-7a92e929bdde admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:58.981183 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839afc80-ff04-4a61-a09d-7a92e929bdde admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:59.091745 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 873/1738] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 08:41:58 2022] POST /identity/v3/users => generated 287 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 08:41:59.092972 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc5fd57-d0ed-44b7-aee4-37a8b1b48407 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:59.103258 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc5fd57-d0ed-44b7-aee4-37a8b1b48407 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:59.108164 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c82b47-0db7-4eb6-9dd4-40e722eb3869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:59.108775 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c82b47-0db7-4eb6-9dd4-40e722eb3869 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:59.109088 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c82b47-0db7-4eb6-9dd4-40e722eb3869 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:59.109361 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c82b47-0db7-4eb6-9dd4-40e722eb3869 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:59.110018 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c82b47-0db7-4eb6-9dd4-40e722eb3869 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:59.112196 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c82b47-0db7-4eb6-9dd4-40e722eb3869 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:59.124219 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 866/1739] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:41:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:41:59.129483 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839afc80-ff04-4a61-a09d-7a92e929bdde admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:59.140140 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839afc80-ff04-4a61-a09d-7a92e929bdde admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:59.145244 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c04cb5-12e8-4644-9cab-0ecfe3d61035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:59.145907 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c04cb5-12e8-4644-9cab-0ecfe3d61035 admin admin] REQUEST_METHOD: `PUT` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:59.146190 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c04cb5-12e8-4644-9cab-0ecfe3d61035 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:59.146465 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c04cb5-12e8-4644-9cab-0ecfe3d61035 admin admin] PATH_INFO: `/v3/projects/ee0f74de0d234e5388a7c393035dbed2/users/f2414497a93a4dc9989494e105880291/roles/ec3751692d9c4ecf81b4a71c936c46be` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:59.163774 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c04cb5-12e8-4644-9cab-0ecfe3d61035 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee0f74de0d234e5388a7c393035dbed2, user_id=f2414497a93a4dc9989494e105880291, role_id=ec3751692d9c4ecf81b4a71c936c46be)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:59.166535 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c04cb5-12e8-4644-9cab-0ecfe3d61035 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:59.226374 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 874/1740] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 08:41:59 2022] PUT /identity/v3/projects/ee0f74de0d234e5388a7c393035dbed2/users/f2414497a93a4dc9989494e105880291/roles/ec3751692d9c4ecf81b4a71c936c46be => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:41:59.230621 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c82b47-0db7-4eb6-9dd4-40e722eb3869 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:59.241014 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c82b47-0db7-4eb6-9dd4-40e722eb3869 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:59.299965 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7580e3-2ae5-46d6-ac0d-f008d6d30d7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:59.300581 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7580e3-2ae5-46d6-ac0d-f008d6d30d7d admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:59.300863 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7580e3-2ae5-46d6-ac0d-f008d6d30d7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:59.301162 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7580e3-2ae5-46d6-ac0d-f008d6d30d7d admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:59.301925 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7580e3-2ae5-46d6-ac0d-f008d6d30d7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:59.303687 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7580e3-2ae5-46d6-ac0d-f008d6d30d7d admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:59.315144 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 867/1741] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:41:59 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:41:59.320712 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c04cb5-12e8-4644-9cab-0ecfe3d61035 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:59.331133 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c04cb5-12e8-4644-9cab-0ecfe3d61035 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:59.344133 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2944c24-1112-43b2-a8ff-f6108954c5ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:59.344797 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2944c24-1112-43b2-a8ff-f6108954c5ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:59.345129 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2944c24-1112-43b2-a8ff-f6108954c5ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:59.345396 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2944c24-1112-43b2-a8ff-f6108954c5ef admin admin] PATH_INFO: `/v3/projects/ee0f74de0d234e5388a7c393035dbed2/users/f2414497a93a4dc9989494e105880291/roles/ec3751692d9c4ecf81b4a71c936c46be` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:59.347487 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2944c24-1112-43b2-a8ff-f6108954c5ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee0f74de0d234e5388a7c393035dbed2, user_id=f2414497a93a4dc9989494e105880291, role_id=ec3751692d9c4ecf81b4a71c936c46be)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:59.351413 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2944c24-1112-43b2-a8ff-f6108954c5ef admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:59.361888 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 875/1742] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 08:41:59 2022] PUT /identity/v3/projects/ee0f74de0d234e5388a7c393035dbed2/users/f2414497a93a4dc9989494e105880291/roles/ec3751692d9c4ecf81b4a71c936c46be => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:41:59.373006 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778eadd2-c330-4186-af40-b33073d7eec0 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:59.373606 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778eadd2-c330-4186-af40-b33073d7eec0 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:59.373888 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778eadd2-c330-4186-af40-b33073d7eec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:59.408206 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-778eadd2-c330-4186-af40-b33073d7eec0 None None] MFA Rules not processed for user `f2414497a93a4dc9989494e105880291`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:41:59.416297 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-778eadd2-c330-4186-af40-b33073d7eec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:41:59.499097 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:41:59.526015 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 868/1743] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:41:59 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:41:59.543889 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2944c24-1112-43b2-a8ff-f6108954c5ef admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:41:59.545878 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-c2944c24-1112-43b2-a8ff-f6108954c5ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:41:59.631656 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2944c24-1112-43b2-a8ff-f6108954c5ef admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:41:59.635528 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac0fe06-50ec-4163-b47f-4032c552980f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ba74e28ced4cbba857f54222dfbe41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:41:59.636282 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac0fe06-50ec-4163-b47f-4032c552980f service cinder] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:41:59.636644 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac0fe06-50ec-4163-b47f-4032c552980f service cinder] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:41:59.636979 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac0fe06-50ec-4163-b47f-4032c552980f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:41:59.647486 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac0fe06-50ec-4163-b47f-4032c552980f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:41:59.649101 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac0fe06-50ec-4163-b47f-4032c552980f service cinder] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:41:59.665132 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 876/1744] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:41:59 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:42:00.658917 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778eadd2-c330-4186-af40-b33073d7eec0 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:00.850585 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778eadd2-c330-4186-af40-b33073d7eec0 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:00.850585 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd471113-de54-4eaa-9c80-8dff09b8bcea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:00.850585 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd471113-de54-4eaa-9c80-8dff09b8bcea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:00.850585 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd471113-de54-4eaa-9c80-8dff09b8bcea admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:00.851360 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd471113-de54-4eaa-9c80-8dff09b8bcea admin admin] PATH_INFO: `/v3/users/600bbed27a5344f9ad14ffc91ea2b1d7` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:00.851360 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd471113-de54-4eaa-9c80-8dff09b8bcea admin admin] RBAC: Authorizing `identity:delete_user(user_id=600bbed27a5344f9ad14ffc91ea2b1d7)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:00.851360 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd471113-de54-4eaa-9c80-8dff09b8bcea admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:01.178090 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65a6fe6-7033-42e6-80b1-98571d89f9ef None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:01.178090 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65a6fe6-7033-42e6-80b1-98571d89f9ef None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:01.178090 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65a6fe6-7033-42e6-80b1-98571d89f9ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:01.231436 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-a65a6fe6-7033-42e6-80b1-98571d89f9ef None None] MFA Rules not processed for user `d4863ecff21e408c9deca645b88d3727`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:42:01.234192 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-a65a6fe6-7033-42e6-80b1-98571d89f9ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:42:01.348816 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-dd471113-de54-4eaa-9c80-8dff09b8bcea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '600bbed27a5344f9ad14ffc91ea2b1d7'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:01.351868 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 877/1745] 10.222.0.59 () {58 vars in 1015 bytes} [Wed Aug 10 08:42:01 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 08:42:01.369960 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-dd471113-de54-4eaa-9c80-8dff09b8bcea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '600bbed27a5344f9ad14ffc91ea2b1d7'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:01.390057 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-dd471113-de54-4eaa-9c80-8dff09b8bcea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '600bbed27a5344f9ad14ffc91ea2b1d7'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:01.393510 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-dd471113-de54-4eaa-9c80-8dff09b8bcea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '600bbed27a5344f9ad14ffc91ea2b1d7'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:01.395857 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-dd471113-de54-4eaa-9c80-8dff09b8bcea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '600bbed27a5344f9ad14ffc91ea2b1d7'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:01.479684 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 869/1746] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 08:42:00 2022] DELETE /identity/v3/users/600bbed27a5344f9ad14ffc91ea2b1d7 => generated 0 bytes in 821 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:42:01.480489 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd869cc8-303e-4e1a-8a7f-325b451aaa35 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:01.480778 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd869cc8-303e-4e1a-8a7f-325b451aaa35 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:01.481076 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd869cc8-303e-4e1a-8a7f-325b451aaa35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:01.550908 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-bd869cc8-303e-4e1a-8a7f-325b451aaa35 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:42:01.552331 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-bd869cc8-303e-4e1a-8a7f-325b451aaa35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:42:01.595863 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:42:01.667461 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd471113-de54-4eaa-9c80-8dff09b8bcea admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:01.670423 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-dd471113-de54-4eaa-9c80-8dff09b8bcea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:42:01.695375 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 878/1747] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:42:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:42:01.713499 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd869cc8-303e-4e1a-8a7f-325b451aaa35 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:01.717873 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-bd869cc8-303e-4e1a-8a7f-325b451aaa35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:42:01.820105 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd471113-de54-4eaa-9c80-8dff09b8bcea admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:01.832400 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7510cddd-6672-47d4-8ee8-7ef7552f1f64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:01.834113 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7510cddd-6672-47d4-8ee8-7ef7552f1f64 service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:01.834113 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7510cddd-6672-47d4-8ee8-7ef7552f1f64 service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:01.834113 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7510cddd-6672-47d4-8ee8-7ef7552f1f64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:01.838380 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-7510cddd-6672-47d4-8ee8-7ef7552f1f64 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:42:01.863528 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7510cddd-6672-47d4-8ee8-7ef7552f1f64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:01.864990 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7510cddd-6672-47d4-8ee8-7ef7552f1f64 service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:01.882672 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 870/1748] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:42:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 08:42:01.891797 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd869cc8-303e-4e1a-8a7f-325b451aaa35 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:01.902354 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730e9850-b7ac-4ba2-bdda-2e7c382420b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:01.903581 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730e9850-b7ac-4ba2-bdda-2e7c382420b3 service neutron] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:01.904711 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730e9850-b7ac-4ba2-bdda-2e7c382420b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:01.905126 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730e9850-b7ac-4ba2-bdda-2e7c382420b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:01.925217 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730e9850-b7ac-4ba2-bdda-2e7c382420b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:01.927048 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730e9850-b7ac-4ba2-bdda-2e7c382420b3 service neutron] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:01.941369 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 879/1749] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:42:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:42:03.727309 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7510cddd-6672-47d4-8ee8-7ef7552f1f64 service neutron] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:03.743458 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-7510cddd-6672-47d4-8ee8-7ef7552f1f64 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:42:03.781151 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7510cddd-6672-47d4-8ee8-7ef7552f1f64 service neutron] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:03.792462 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0e0d35-5af2-4ce9-95b0-0d40f181af40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:03.795235 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0e0d35-5af2-4ce9-95b0-0d40f181af40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:03.796101 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0e0d35-5af2-4ce9-95b0-0d40f181af40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:03.798386 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0e0d35-5af2-4ce9-95b0-0d40f181af40 admin admin] PATH_INFO: `/v3/projects/17abac2aed18465c85d6ad6b8bd2d7f7` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:03.802295 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0e0d35-5af2-4ce9-95b0-0d40f181af40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=17abac2aed18465c85d6ad6b8bd2d7f7)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:03.806964 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0e0d35-5af2-4ce9-95b0-0d40f181af40 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:03.843402 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:42:04.082118 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-6a0e0d35-5af2-4ce9-95b0-0d40f181af40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17abac2aed18465c85d6ad6b8bd2d7f7', 'request_id': 'req-6a0e0d35-5af2-4ce9-95b0-0d40f181af40'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:04.157245 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-6a0e0d35-5af2-4ce9-95b0-0d40f181af40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17abac2aed18465c85d6ad6b8bd2d7f7', 'request_id': 'req-6a0e0d35-5af2-4ce9-95b0-0d40f181af40'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:04.165131 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-6a0e0d35-5af2-4ce9-95b0-0d40f181af40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17abac2aed18465c85d6ad6b8bd2d7f7', 'request_id': 'req-6a0e0d35-5af2-4ce9-95b0-0d40f181af40'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:04.193924 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-6a0e0d35-5af2-4ce9-95b0-0d40f181af40 admin admin] The token cache is being invalidate because project 17abac2aed18465c85d6ad6b8bd2d7f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 08:42:04.194218 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-6a0e0d35-5af2-4ce9-95b0-0d40f181af40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:04.201446 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-6a0e0d35-5af2-4ce9-95b0-0d40f181af40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:04.210297 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 871/1750] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 08:42:03 2022] DELETE /identity/v3/projects/17abac2aed18465c85d6ad6b8bd2d7f7 => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:42:04.216111 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730e9850-b7ac-4ba2-bdda-2e7c382420b3 service neutron] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:04.261153 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730e9850-b7ac-4ba2-bdda-2e7c382420b3 service neutron] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:04.486394 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad06073-005a-4898-9d32-2d3870f988a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:04.487031 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad06073-005a-4898-9d32-2d3870f988a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:04.489563 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad06073-005a-4898-9d32-2d3870f988a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:04.489563 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad06073-005a-4898-9d32-2d3870f988a6 admin admin] PATH_INFO: `/v3/users/f2414497a93a4dc9989494e105880291` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:04.489563 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad06073-005a-4898-9d32-2d3870f988a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2414497a93a4dc9989494e105880291)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:04.491183 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad06073-005a-4898-9d32-2d3870f988a6 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:04.807230 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-9ad06073-005a-4898-9d32-2d3870f988a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2414497a93a4dc9989494e105880291'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:04.838234 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-9ad06073-005a-4898-9d32-2d3870f988a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2414497a93a4dc9989494e105880291'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:04.840480 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:42:04.848297 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-9ad06073-005a-4898-9d32-2d3870f988a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2414497a93a4dc9989494e105880291'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:04.945412 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-9ad06073-005a-4898-9d32-2d3870f988a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2414497a93a4dc9989494e105880291'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:04.950254 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-9ad06073-005a-4898-9d32-2d3870f988a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2414497a93a4dc9989494e105880291'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:04.956177 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 880/1751] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 08:42:04 2022] DELETE /identity/v3/users/f2414497a93a4dc9989494e105880291 => generated 0 bytes in 742 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:42:05.295648 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66e055c-673f-4fd0-8448-6f1eb451c3e2 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:05.296205 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66e055c-673f-4fd0-8448-6f1eb451c3e2 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:05.296311 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66e055c-673f-4fd0-8448-6f1eb451c3e2 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:05.297820 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 872/1752] 10.222.0.57 () {54 vars in 921 bytes} [Wed Aug 10 08:42:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:42:05.333302 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088106af-3402-4993-b163-bea3555c56c2 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:05.333918 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088106af-3402-4993-b163-bea3555c56c2 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:05.334982 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088106af-3402-4993-b163-bea3555c56c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:05.414370 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-088106af-3402-4993-b163-bea3555c56c2 None None] MFA Rules not processed for user `a943008193914f76a663d489bd73e873`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:42:05.415602 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-088106af-3402-4993-b163-bea3555c56c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:42:05.582789 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 881/1753] 10.222.0.57 () {58 vars in 1018 bytes} [Wed Aug 10 08:42:05 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:42:05.623275 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66e055c-673f-4fd0-8448-6f1eb451c3e2 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:05.626991 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-f66e055c-673f-4fd0-8448-6f1eb451c3e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:42:05.817972 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66e055c-673f-4fd0-8448-6f1eb451c3e2 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:05.827729 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eea2a41-a227-42a5-a3b3-c8010f83a4cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:05.828908 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eea2a41-a227-42a5-a3b3-c8010f83a4cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:05.829679 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eea2a41-a227-42a5-a3b3-c8010f83a4cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:05.831600 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eea2a41-a227-42a5-a3b3-c8010f83a4cb admin admin] PATH_INFO: `/v3/users/ecb975b02b4447479aa36f7ccc9c474a` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:05.832180 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eea2a41-a227-42a5-a3b3-c8010f83a4cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=ecb975b02b4447479aa36f7ccc9c474a)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:05.839777 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eea2a41-a227-42a5-a3b3-c8010f83a4cb admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:05.923685 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088106af-3402-4993-b163-bea3555c56c2 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:05.926766 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-088106af-3402-4993-b163-bea3555c56c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:42:05.952269 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088106af-3402-4993-b163-bea3555c56c2 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:05.971299 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c25c1cc-a0f1-42d5-b7c0-2a569be30cb6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a943008193914f76a663d489bd73e873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:05.972387 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c25c1cc-a0f1-42d5-b7c0-2a569be30cb6 service nova] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:05.972752 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c25c1cc-a0f1-42d5-b7c0-2a569be30cb6 service nova] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:05.973774 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c25c1cc-a0f1-42d5-b7c0-2a569be30cb6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:05.986356 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c25c1cc-a0f1-42d5-b7c0-2a569be30cb6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:05.989075 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c25c1cc-a0f1-42d5-b7c0-2a569be30cb6 service nova] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:06.010426 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 882/1754] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:42:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:42:06.039201 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c25c1cc-a0f1-42d5-b7c0-2a569be30cb6 service nova] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:06.042973 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-5c25c1cc-a0f1-42d5-b7c0-2a569be30cb6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:42:06.072266 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c25c1cc-a0f1-42d5-b7c0-2a569be30cb6 service nova] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:06.082741 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49348817-a588-4c53-ac25-53c3a600ef15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:06.083431 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49348817-a588-4c53-ac25-53c3a600ef15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:06.083806 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49348817-a588-4c53-ac25-53c3a600ef15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:06.084085 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49348817-a588-4c53-ac25-53c3a600ef15 admin admin] PATH_INFO: `/v3/users/24b90fa3db904d76aea127f094d2fd9f` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:06.085586 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49348817-a588-4c53-ac25-53c3a600ef15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=24b90fa3db904d76aea127f094d2fd9f)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:06.087599 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49348817-a588-4c53-ac25-53c3a600ef15 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:06.104643 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-7eea2a41-a227-42a5-a3b3-c8010f83a4cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ecb975b02b4447479aa36f7ccc9c474a'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:06.136147 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-7eea2a41-a227-42a5-a3b3-c8010f83a4cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ecb975b02b4447479aa36f7ccc9c474a'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:06.155120 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-7eea2a41-a227-42a5-a3b3-c8010f83a4cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ecb975b02b4447479aa36f7ccc9c474a'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:06.155938 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-7eea2a41-a227-42a5-a3b3-c8010f83a4cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ecb975b02b4447479aa36f7ccc9c474a'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:06.156553 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-7eea2a41-a227-42a5-a3b3-c8010f83a4cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ecb975b02b4447479aa36f7ccc9c474a'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:06.253712 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 873/1755] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 08:42:05 2022] DELETE /identity/v3/users/ecb975b02b4447479aa36f7ccc9c474a => generated 0 bytes in 628 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:42:06.256201 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5472191d-ba34-475a-918f-21a5fc51e88c None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:06.257717 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5472191d-ba34-475a-918f-21a5fc51e88c None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:06.258364 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5472191d-ba34-475a-918f-21a5fc51e88c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:06.297222 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-5472191d-ba34-475a-918f-21a5fc51e88c None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:42:06.300062 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-5472191d-ba34-475a-918f-21a5fc51e88c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:42:06.312088 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-49348817-a588-4c53-ac25-53c3a600ef15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24b90fa3db904d76aea127f094d2fd9f'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:06.318947 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:42:06.354559 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-49348817-a588-4c53-ac25-53c3a600ef15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24b90fa3db904d76aea127f094d2fd9f'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:06.375323 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-49348817-a588-4c53-ac25-53c3a600ef15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24b90fa3db904d76aea127f094d2fd9f'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:06.422843 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 874/1756] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:42:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:42:06.443009 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-49348817-a588-4c53-ac25-53c3a600ef15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24b90fa3db904d76aea127f094d2fd9f'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:06.443928 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-49348817-a588-4c53-ac25-53c3a600ef15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24b90fa3db904d76aea127f094d2fd9f'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:06.454470 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 883/1757] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 08:42:06 2022] DELETE /identity/v3/users/24b90fa3db904d76aea127f094d2fd9f => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 08:42:06.460123 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e5646a-8761-4e5a-a706-9b42c50047d9 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:06.460464 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e5646a-8761-4e5a-a706-9b42c50047d9 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:06.460853 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e5646a-8761-4e5a-a706-9b42c50047d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:06.493882 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-08e5646a-8761-4e5a-a706-9b42c50047d9 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:42:06.496427 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-08e5646a-8761-4e5a-a706-9b42c50047d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:42:06.498847 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49348817-a588-4c53-ac25-53c3a600ef15 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:06.502177 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-49348817-a588-4c53-ac25-53c3a600ef15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:42:06.607227 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:42:06.644657 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49348817-a588-4c53-ac25-53c3a600ef15 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:06.651953 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385347b3-8fbb-4e7f-b803-86b0d741ef01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:06.653699 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385347b3-8fbb-4e7f-b803-86b0d741ef01 service neutron] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:06.654179 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385347b3-8fbb-4e7f-b803-86b0d741ef01 service neutron] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:06.654642 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385347b3-8fbb-4e7f-b803-86b0d741ef01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:06.667714 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-385347b3-8fbb-4e7f-b803-86b0d741ef01 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:42:06.680767 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385347b3-8fbb-4e7f-b803-86b0d741ef01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:06.682645 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385347b3-8fbb-4e7f-b803-86b0d741ef01 service neutron] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:06.687026 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 875/1758] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:42:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:42:06.698020 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 884/1759] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:42:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:42:07.016024 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e5646a-8761-4e5a-a706-9b42c50047d9 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:07.016024 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e5646a-8761-4e5a-a706-9b42c50047d9 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:07.016024 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a819d3-d93d-462a-b18f-1eb925d588e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:07.016024 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a819d3-d93d-462a-b18f-1eb925d588e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:07.018009 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a819d3-d93d-462a-b18f-1eb925d588e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:07.018009 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a819d3-d93d-462a-b18f-1eb925d588e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:07.018009 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a819d3-d93d-462a-b18f-1eb925d588e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:07.018009 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a819d3-d93d-462a-b18f-1eb925d588e5 service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:07.018009 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 876/1760] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:42:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:42:07.774979 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385347b3-8fbb-4e7f-b803-86b0d741ef01 service neutron] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:07.777364 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-385347b3-8fbb-4e7f-b803-86b0d741ef01 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:42:07.790808 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385347b3-8fbb-4e7f-b803-86b0d741ef01 service neutron] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:07.804129 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b340be8e-ced2-434e-8266-e9650d0de115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:07.806555 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b340be8e-ced2-434e-8266-e9650d0de115 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:07.807319 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b340be8e-ced2-434e-8266-e9650d0de115 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:07.807697 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b340be8e-ced2-434e-8266-e9650d0de115 admin admin] PATH_INFO: `/v3/projects/ee0f74de0d234e5388a7c393035dbed2` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:07.812132 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b340be8e-ced2-434e-8266-e9650d0de115 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee0f74de0d234e5388a7c393035dbed2)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:07.815050 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b340be8e-ced2-434e-8266-e9650d0de115 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:07.969265 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-b340be8e-ced2-434e-8266-e9650d0de115 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee0f74de0d234e5388a7c393035dbed2', 'request_id': 'req-b340be8e-ced2-434e-8266-e9650d0de115'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:07.979306 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-b340be8e-ced2-434e-8266-e9650d0de115 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee0f74de0d234e5388a7c393035dbed2', 'request_id': 'req-b340be8e-ced2-434e-8266-e9650d0de115'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:08.089653 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93541929-5139-42ad-8609-b5a17eccd759 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:08.090635 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93541929-5139-42ad-8609-b5a17eccd759 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:08.091150 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93541929-5139-42ad-8609-b5a17eccd759 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:08.093108 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 877/1761] 10.222.0.57 () {54 vars in 921 bytes} [Wed Aug 10 08:42:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 08:42:08.111792 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77f620f-6b2e-4ce8-ba46-9ab5724704da None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:08.112378 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77f620f-6b2e-4ce8-ba46-9ab5724704da None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:08.112881 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77f620f-6b2e-4ce8-ba46-9ab5724704da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:08.117356 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-b340be8e-ced2-434e-8266-e9650d0de115 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee0f74de0d234e5388a7c393035dbed2', 'request_id': 'req-b340be8e-ced2-434e-8266-e9650d0de115'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:08.143173 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-b340be8e-ced2-434e-8266-e9650d0de115 admin admin] The token cache is being invalidate because project ee0f74de0d234e5388a7c393035dbed2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 08:42:08.143734 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-b340be8e-ced2-434e-8266-e9650d0de115 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:08.144834 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-b340be8e-ced2-434e-8266-e9650d0de115 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:08.150675 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 885/1762] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 08:42:07 2022] DELETE /identity/v3/projects/ee0f74de0d234e5388a7c393035dbed2 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:42:08.169510 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743b5f5c-c6ca-4e20-a2ca-5825237e1d01 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:08.169510 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743b5f5c-c6ca-4e20-a2ca-5825237e1d01 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:08.169510 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743b5f5c-c6ca-4e20-a2ca-5825237e1d01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:08.239346 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-e77f620f-6b2e-4ce8-ba46-9ab5724704da None None] MFA Rules not processed for user `a943008193914f76a663d489bd73e873`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:42:08.242369 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-743b5f5c-c6ca-4e20-a2ca-5825237e1d01 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:42:08.242526 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-e77f620f-6b2e-4ce8-ba46-9ab5724704da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:42:08.247668 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-743b5f5c-c6ca-4e20-a2ca-5825237e1d01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:42:08.408160 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 878/1763] 10.222.0.57 () {58 vars in 1018 bytes} [Wed Aug 10 08:42:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:42:08.427498 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 886/1764] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:42:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:42:08.429430 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db0ae75-bae4-4a60-8224-9ac529986a48 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:08.429726 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db0ae75-bae4-4a60-8224-9ac529986a48 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:08.430003 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db0ae75-bae4-4a60-8224-9ac529986a48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:08.471368 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-0db0ae75-bae4-4a60-8224-9ac529986a48 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:42:08.473674 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-0db0ae75-bae4-4a60-8224-9ac529986a48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:42:08.500020 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 879/1765] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:42:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:42:08.502258 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743b5f5c-c6ca-4e20-a2ca-5825237e1d01 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:08.519042 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743b5f5c-c6ca-4e20-a2ca-5825237e1d01 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:08.538599 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909c6691-f233-48d0-aa99-02a745a72595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:08.539387 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909c6691-f233-48d0-aa99-02a745a72595 admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:08.539776 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909c6691-f233-48d0-aa99-02a745a72595 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:08.540168 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909c6691-f233-48d0-aa99-02a745a72595 admin admin] PATH_INFO: `/v3/domains` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:08.541017 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909c6691-f233-48d0-aa99-02a745a72595 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:42:08.541476 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909c6691-f233-48d0-aa99-02a745a72595 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:08.543198 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909c6691-f233-48d0-aa99-02a745a72595 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:08.576978 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 887/1766] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Aug 10 08:42:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:42:08.577867 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db0ae75-bae4-4a60-8224-9ac529986a48 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:08.596941 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db0ae75-bae4-4a60-8224-9ac529986a48 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:08.607079 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b6529e-cf6d-41a8-b6c8-857772890a1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:08.609317 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b6529e-cf6d-41a8-b6c8-857772890a1d admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:08.609317 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b6529e-cf6d-41a8-b6c8-857772890a1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:08.609317 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b6529e-cf6d-41a8-b6c8-857772890a1d admin admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:08.616470 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b6529e-cf6d-41a8-b6c8-857772890a1d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:08.621755 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b6529e-cf6d-41a8-b6c8-857772890a1d admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:08.721748 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 880/1767] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 08:42:08 2022] POST /identity/v3/projects => generated 377 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:42:08.727298 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909c6691-f233-48d0-aa99-02a745a72595 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:08.738650 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909c6691-f233-48d0-aa99-02a745a72595 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:08.806763 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:42:08.826182 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f798e2d-9ea5-4b40-a706-d1d4b0d7d69b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:08.826849 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f798e2d-9ea5-4b40-a706-d1d4b0d7d69b admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:08.828391 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f798e2d-9ea5-4b40-a706-d1d4b0d7d69b admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:08.828391 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f798e2d-9ea5-4b40-a706-d1d4b0d7d69b admin admin] PATH_INFO: `/v3/users` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:08.828699 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f798e2d-9ea5-4b40-a706-d1d4b0d7d69b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:08.831433 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f798e2d-9ea5-4b40-a706-d1d4b0d7d69b admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:08.947543 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 888/1768] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 08:42:08 2022] POST /identity/v3/users => generated 289 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:42:08.951656 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b6529e-cf6d-41a8-b6c8-857772890a1d admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:08.963604 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:42:08.973661 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b6529e-cf6d-41a8-b6c8-857772890a1d admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:08.979041 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8461d649-6365-40ea-bc50-bcee708c976d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:08.979989 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8461d649-6365-40ea-bc50-bcee708c976d admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:08.980457 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8461d649-6365-40ea-bc50-bcee708c976d admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:08.981043 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8461d649-6365-40ea-bc50-bcee708c976d admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:08.982414 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8461d649-6365-40ea-bc50-bcee708c976d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:08.985266 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8461d649-6365-40ea-bc50-bcee708c976d admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:09.008403 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 881/1769] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:42:08 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:42:09.009205 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f798e2d-9ea5-4b40-a706-d1d4b0d7d69b admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:09.022803 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f798e2d-9ea5-4b40-a706-d1d4b0d7d69b admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:09.031357 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39c5bdf-33a2-4459-b64b-98afd03e41f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:09.033192 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39c5bdf-33a2-4459-b64b-98afd03e41f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:09.047642 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39c5bdf-33a2-4459-b64b-98afd03e41f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:09.053299 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39c5bdf-33a2-4459-b64b-98afd03e41f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:09.053299 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39c5bdf-33a2-4459-b64b-98afd03e41f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:09.053299 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39c5bdf-33a2-4459-b64b-98afd03e41f5 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:09.068159 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 889/1770] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:42:09 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 08:42:09.069518 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8461d649-6365-40ea-bc50-bcee708c976d admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:09.082477 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8461d649-6365-40ea-bc50-bcee708c976d admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:09.089706 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2f65af-6207-4b6d-969f-1ade19be3196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:09.091658 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2f65af-6207-4b6d-969f-1ade19be3196 admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:09.092659 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2f65af-6207-4b6d-969f-1ade19be3196 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:09.093293 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2f65af-6207-4b6d-969f-1ade19be3196 admin admin] PATH_INFO: `/v3/projects/f20c187d7b6d411cba09bf374ccc5390/users/e07a2f0f9d1f4bdb82e3c8cd87cba512/roles/44b64db484f24a46bf0b7ced46b519bf` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:09.112149 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2f65af-6207-4b6d-969f-1ade19be3196 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f20c187d7b6d411cba09bf374ccc5390, user_id=e07a2f0f9d1f4bdb82e3c8cd87cba512, role_id=44b64db484f24a46bf0b7ced46b519bf)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:09.120729 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2f65af-6207-4b6d-969f-1ade19be3196 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:09.202727 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 882/1771] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 08:42:09 2022] PUT /identity/v3/projects/f20c187d7b6d411cba09bf374ccc5390/users/e07a2f0f9d1f4bdb82e3c8cd87cba512/roles/44b64db484f24a46bf0b7ced46b519bf => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:42:09.203948 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fa7b64-ab88-40d3-8b0b-f0eb0a10c316 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:09.205401 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fa7b64-ab88-40d3-8b0b-f0eb0a10c316 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:09.205401 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fa7b64-ab88-40d3-8b0b-f0eb0a10c316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:09.256746 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-a4fa7b64-ab88-40d3-8b0b-f0eb0a10c316 None None] MFA Rules not processed for user `e07a2f0f9d1f4bdb82e3c8cd87cba512`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:42:09.258134 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-a4fa7b64-ab88-40d3-8b0b-f0eb0a10c316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:42:09.379009 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 890/1772] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:42:09 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 08:42:09.383323 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2f65af-6207-4b6d-969f-1ade19be3196 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:09.494442 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2f65af-6207-4b6d-969f-1ade19be3196 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:09.500546 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dbe586-4a51-495c-b9db-17c4eb2b39b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:09.502089 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dbe586-4a51-495c-b9db-17c4eb2b39b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:09.502494 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dbe586-4a51-495c-b9db-17c4eb2b39b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:09.502869 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dbe586-4a51-495c-b9db-17c4eb2b39b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:09.503611 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dbe586-4a51-495c-b9db-17c4eb2b39b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:09.506069 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dbe586-4a51-495c-b9db-17c4eb2b39b7 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:09.585021 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 883/1773] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 08:42:09 2022] POST /identity/v3/projects => generated 377 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:42:09.593794 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fa7b64-ab88-40d3-8b0b-f0eb0a10c316 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:09.616277 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fa7b64-ab88-40d3-8b0b-f0eb0a10c316 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:09.755671 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c4f36f-ee5f-47a3-90b9-c04d456b2ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:09.759108 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c4f36f-ee5f-47a3-90b9-c04d456b2ea7 admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:09.759108 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c4f36f-ee5f-47a3-90b9-c04d456b2ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:09.759108 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c4f36f-ee5f-47a3-90b9-c04d456b2ea7 admin admin] PATH_INFO: `/v3/users` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:09.759108 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c4f36f-ee5f-47a3-90b9-c04d456b2ea7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:09.760956 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c4f36f-ee5f-47a3-90b9-c04d456b2ea7 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:09.854310 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dbe586-4a51-495c-b9db-17c4eb2b39b7 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:09.862977 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-c4dbe586-4a51-495c-b9db-17c4eb2b39b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:42:09.875206 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 891/1774] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 08:42:09 2022] POST /identity/v3/users => generated 295 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:42:09.878763 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c4f36f-ee5f-47a3-90b9-c04d456b2ea7 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:09.894473 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c4f36f-ee5f-47a3-90b9-c04d456b2ea7 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:09.901935 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e813296-97bf-4174-8bcf-87595484bd70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:09.902897 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e813296-97bf-4174-8bcf-87595484bd70 admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:09.903421 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e813296-97bf-4174-8bcf-87595484bd70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:09.903967 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e813296-97bf-4174-8bcf-87595484bd70 admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:09.905219 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e813296-97bf-4174-8bcf-87595484bd70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:09.907963 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e813296-97bf-4174-8bcf-87595484bd70 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:09.928979 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 892/1775] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:42:09 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:42:09.933203 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e813296-97bf-4174-8bcf-87595484bd70 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:09.957433 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e813296-97bf-4174-8bcf-87595484bd70 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:09.967156 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ab981c-102f-44ba-ae20-7157222c725b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:09.967156 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ab981c-102f-44ba-ae20-7157222c725b admin admin] REQUEST_METHOD: `PUT` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:09.967156 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ab981c-102f-44ba-ae20-7157222c725b admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:09.967156 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ab981c-102f-44ba-ae20-7157222c725b admin admin] PATH_INFO: `/v3/projects/73aeab67d55e4bb284a1ca2e14aab780/users/246efda3b947486a8c054a0e6583080a/roles/ec3751692d9c4ecf81b4a71c936c46be` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:09.996154 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dbe586-4a51-495c-b9db-17c4eb2b39b7 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:10.003728 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4208bc81-8a4f-4adc-ac14-2e393753dc34 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a943008193914f76a663d489bd73e873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:10.004650 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4208bc81-8a4f-4adc-ac14-2e393753dc34 service nova] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:10.004807 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4208bc81-8a4f-4adc-ac14-2e393753dc34 service nova] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:10.005147 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4208bc81-8a4f-4adc-ac14-2e393753dc34 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:10.018666 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ab981c-102f-44ba-ae20-7157222c725b admin admin] RBAC: Authorizing `identity:create_grant(project_id=73aeab67d55e4bb284a1ca2e14aab780, user_id=246efda3b947486a8c054a0e6583080a, role_id=ec3751692d9c4ecf81b4a71c936c46be)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:10.018892 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4208bc81-8a4f-4adc-ac14-2e393753dc34 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:10.020380 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4208bc81-8a4f-4adc-ac14-2e393753dc34 service nova] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:10.025368 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ab981c-102f-44ba-ae20-7157222c725b admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:10.064094 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 884/1776] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:42:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 08:42:10.201173 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 893/1777] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 08:42:09 2022] PUT /identity/v3/projects/73aeab67d55e4bb284a1ca2e14aab780/users/246efda3b947486a8c054a0e6583080a/roles/ec3751692d9c4ecf81b4a71c936c46be => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 08:42:10.208124 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4208bc81-8a4f-4adc-ac14-2e393753dc34 service nova] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:10.236187 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4208bc81-8a4f-4adc-ac14-2e393753dc34 service nova] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:10.297629 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:42:10.348700 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c9cd98-9af9-4e52-a3cc-3da5495982d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:10.349875 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c9cd98-9af9-4e52-a3cc-3da5495982d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:10.352375 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c9cd98-9af9-4e52-a3cc-3da5495982d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:10.352762 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c9cd98-9af9-4e52-a3cc-3da5495982d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:10.353848 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c9cd98-9af9-4e52-a3cc-3da5495982d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:10.356491 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c9cd98-9af9-4e52-a3cc-3da5495982d1 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:10.380714 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 885/1778] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:42:10 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:42:10.384537 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ab981c-102f-44ba-ae20-7157222c725b admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:10.401710 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ab981c-102f-44ba-ae20-7157222c725b admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:10.420198 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476df5e6-4fee-49b9-bdc7-61a2830864fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:10.420676 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476df5e6-4fee-49b9-bdc7-61a2830864fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:10.421116 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476df5e6-4fee-49b9-bdc7-61a2830864fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:10.421457 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476df5e6-4fee-49b9-bdc7-61a2830864fc admin admin] PATH_INFO: `/v3/projects/73aeab67d55e4bb284a1ca2e14aab780/users/246efda3b947486a8c054a0e6583080a/roles/ec3751692d9c4ecf81b4a71c936c46be` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:10.426396 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476df5e6-4fee-49b9-bdc7-61a2830864fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=73aeab67d55e4bb284a1ca2e14aab780, user_id=246efda3b947486a8c054a0e6583080a, role_id=ec3751692d9c4ecf81b4a71c936c46be)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:10.435990 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476df5e6-4fee-49b9-bdc7-61a2830864fc admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:10.446199 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c9cd98-9af9-4e52-a3cc-3da5495982d1 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:10.449694 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:42:10.453781 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-91c9cd98-9af9-4e52-a3cc-3da5495982d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:42:10.460632 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 894/1779] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 08:42:10 2022] PUT /identity/v3/projects/73aeab67d55e4bb284a1ca2e14aab780/users/246efda3b947486a8c054a0e6583080a/roles/ec3751692d9c4ecf81b4a71c936c46be => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:42:10.478533 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e674642c-374a-4b2f-95b2-e3548c4dd287 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:10.479467 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e674642c-374a-4b2f-95b2-e3548c4dd287 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:10.479853 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e674642c-374a-4b2f-95b2-e3548c4dd287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:10.490182 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c9cd98-9af9-4e52-a3cc-3da5495982d1 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:10.557441 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-e674642c-374a-4b2f-95b2-e3548c4dd287 None None] MFA Rules not processed for user `246efda3b947486a8c054a0e6583080a`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:42:10.563111 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-e674642c-374a-4b2f-95b2-e3548c4dd287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:42:10.645915 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061ba8de-b10b-4e1c-b52b-56bde9a4b4b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:10.647217 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061ba8de-b10b-4e1c-b52b-56bde9a4b4b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:10.647504 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061ba8de-b10b-4e1c-b52b-56bde9a4b4b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:10.647794 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061ba8de-b10b-4e1c-b52b-56bde9a4b4b7 admin admin] PATH_INFO: `/v3/projects/6acf0b5477e940c08b6b50122a4808bc` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:10.651059 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061ba8de-b10b-4e1c-b52b-56bde9a4b4b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6acf0b5477e940c08b6b50122a4808bc)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:10.659931 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061ba8de-b10b-4e1c-b52b-56bde9a4b4b7 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:10.771236 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 895/1780] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:42:10 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:42:10.772149 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e674642c-374a-4b2f-95b2-e3548c4dd287 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:10.775087 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-e674642c-374a-4b2f-95b2-e3548c4dd287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:42:10.796106 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e674642c-374a-4b2f-95b2-e3548c4dd287 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:10.802402 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e82988d-0cb9-42be-bb30-7bdb07ac9418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:10.803045 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e82988d-0cb9-42be-bb30-7bdb07ac9418 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:10.803332 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e82988d-0cb9-42be-bb30-7bdb07ac9418 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:10.804416 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e82988d-0cb9-42be-bb30-7bdb07ac9418 admin admin] PATH_INFO: `/v3/projects/5c0b83c154c54e7980459d81591f09ee` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:10.806122 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e82988d-0cb9-42be-bb30-7bdb07ac9418 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c0b83c154c54e7980459d81591f09ee)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:10.809336 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e82988d-0cb9-42be-bb30-7bdb07ac9418 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:10.875895 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-061ba8de-b10b-4e1c-b52b-56bde9a4b4b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6acf0b5477e940c08b6b50122a4808bc', 'request_id': 'req-061ba8de-b10b-4e1c-b52b-56bde9a4b4b7'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:10.999573 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-061ba8de-b10b-4e1c-b52b-56bde9a4b4b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6acf0b5477e940c08b6b50122a4808bc', 'request_id': 'req-061ba8de-b10b-4e1c-b52b-56bde9a4b4b7'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:11.025590 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-061ba8de-b10b-4e1c-b52b-56bde9a4b4b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6acf0b5477e940c08b6b50122a4808bc', 'request_id': 'req-061ba8de-b10b-4e1c-b52b-56bde9a4b4b7'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:11.068267 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-9e82988d-0cb9-42be-bb30-7bdb07ac9418 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c0b83c154c54e7980459d81591f09ee', 'request_id': 'req-9e82988d-0cb9-42be-bb30-7bdb07ac9418'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:11.077482 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-9e82988d-0cb9-42be-bb30-7bdb07ac9418 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c0b83c154c54e7980459d81591f09ee', 'request_id': 'req-9e82988d-0cb9-42be-bb30-7bdb07ac9418'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:11.078541 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-061ba8de-b10b-4e1c-b52b-56bde9a4b4b7 admin admin] The token cache is being invalidate because project 6acf0b5477e940c08b6b50122a4808bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 08:42:11.078893 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-061ba8de-b10b-4e1c-b52b-56bde9a4b4b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:11.080507 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-061ba8de-b10b-4e1c-b52b-56bde9a4b4b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:11.085479 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 886/1781] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 08:42:10 2022] DELETE /identity/v3/projects/6acf0b5477e940c08b6b50122a4808bc => generated 0 bytes in 637 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 08:42:11.085479 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061ba8de-b10b-4e1c-b52b-56bde9a4b4b7 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:11.094195 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-061ba8de-b10b-4e1c-b52b-56bde9a4b4b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:42:11.132504 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-9e82988d-0cb9-42be-bb30-7bdb07ac9418 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c0b83c154c54e7980459d81591f09ee', 'request_id': 'req-9e82988d-0cb9-42be-bb30-7bdb07ac9418'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:11.193065 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-9e82988d-0cb9-42be-bb30-7bdb07ac9418 admin admin] The token cache is being invalidate because project 5c0b83c154c54e7980459d81591f09ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 08:42:11.193358 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-9e82988d-0cb9-42be-bb30-7bdb07ac9418 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:11.205725 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-9e82988d-0cb9-42be-bb30-7bdb07ac9418 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:11.219488 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 896/1782] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 08:42:10 2022] DELETE /identity/v3/projects/5c0b83c154c54e7980459d81591f09ee => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:42:11.219488 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e82988d-0cb9-42be-bb30-7bdb07ac9418 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:11.225101 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-9e82988d-0cb9-42be-bb30-7bdb07ac9418 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:42:11.327728 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061ba8de-b10b-4e1c-b52b-56bde9a4b4b7 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:11.366778 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:42:11.432791 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e82988d-0cb9-42be-bb30-7bdb07ac9418 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:11.447343 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad7850b-21b3-442f-9b72-11beccc944ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:11.448002 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad7850b-21b3-442f-9b72-11beccc944ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:11.448281 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad7850b-21b3-442f-9b72-11beccc944ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:11.448559 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad7850b-21b3-442f-9b72-11beccc944ad admin admin] PATH_INFO: `/v3/users/3998061ded274c558f9c38a234a1f184` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:11.451416 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad7850b-21b3-442f-9b72-11beccc944ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=3998061ded274c558f9c38a234a1f184)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:11.453484 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad7850b-21b3-442f-9b72-11beccc944ad admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:11.464743 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880893f7-0639-4395-8dbd-1c9abd1e8001 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ba74e28ced4cbba857f54222dfbe41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:11.465489 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880893f7-0639-4395-8dbd-1c9abd1e8001 service cinder] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:11.465774 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880893f7-0639-4395-8dbd-1c9abd1e8001 service cinder] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:11.466045 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880893f7-0639-4395-8dbd-1c9abd1e8001 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:11.468429 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:42:11.469526 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-880893f7-0639-4395-8dbd-1c9abd1e8001 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:42:11.564664 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880893f7-0639-4395-8dbd-1c9abd1e8001 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:11.566727 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880893f7-0639-4395-8dbd-1c9abd1e8001 service cinder] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:11.588953 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 887/1783] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:42:11 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:42:11.588953 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880893f7-0639-4395-8dbd-1c9abd1e8001 service cinder] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:11.588953 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-880893f7-0639-4395-8dbd-1c9abd1e8001 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:42:11.599326 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880893f7-0639-4395-8dbd-1c9abd1e8001 service cinder] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:11.606177 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cf8b96-14c4-4f54-bfb1-c87da8ef9d9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:11.607032 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cf8b96-14c4-4f54-bfb1-c87da8ef9d9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:11.607412 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cf8b96-14c4-4f54-bfb1-c87da8ef9d9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:11.607801 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cf8b96-14c4-4f54-bfb1-c87da8ef9d9f admin admin] PATH_INFO: `/v3/users/282055067f0143b488116d323c475c7f` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:11.609264 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cf8b96-14c4-4f54-bfb1-c87da8ef9d9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=282055067f0143b488116d323c475c7f)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:11.612771 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cf8b96-14c4-4f54-bfb1-c87da8ef9d9f admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:11.771492 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-6ad7850b-21b3-442f-9b72-11beccc944ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3998061ded274c558f9c38a234a1f184'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:11.799851 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-6ad7850b-21b3-442f-9b72-11beccc944ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3998061ded274c558f9c38a234a1f184'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:11.816755 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-6ad7850b-21b3-442f-9b72-11beccc944ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3998061ded274c558f9c38a234a1f184'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:11.832535 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-61cf8b96-14c4-4f54-bfb1-c87da8ef9d9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '282055067f0143b488116d323c475c7f'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:11.847527 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-61cf8b96-14c4-4f54-bfb1-c87da8ef9d9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '282055067f0143b488116d323c475c7f'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:11.856062 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-61cf8b96-14c4-4f54-bfb1-c87da8ef9d9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '282055067f0143b488116d323c475c7f'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:11.857004 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-61cf8b96-14c4-4f54-bfb1-c87da8ef9d9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '282055067f0143b488116d323c475c7f'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:11.857694 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-61cf8b96-14c4-4f54-bfb1-c87da8ef9d9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '282055067f0143b488116d323c475c7f'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:11.871169 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-6ad7850b-21b3-442f-9b72-11beccc944ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3998061ded274c558f9c38a234a1f184'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:11.871944 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-6ad7850b-21b3-442f-9b72-11beccc944ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3998061ded274c558f9c38a234a1f184'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:11.881701 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 897/1784] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 08:42:11 2022] DELETE /identity/v3/users/3998061ded274c558f9c38a234a1f184 => generated 0 bytes in 664 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:42:11.934156 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 888/1785] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 08:42:11 2022] DELETE /identity/v3/users/282055067f0143b488116d323c475c7f => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:42:12.864591 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad7850b-21b3-442f-9b72-11beccc944ad admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:12.866714 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-6ad7850b-21b3-442f-9b72-11beccc944ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:42:12.953874 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad7850b-21b3-442f-9b72-11beccc944ad admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:13.018514 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5c8cc3-c4d6-454c-abec-94ab44f1fdd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:13.019154 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5c8cc3-c4d6-454c-abec-94ab44f1fdd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:13.019435 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5c8cc3-c4d6-454c-abec-94ab44f1fdd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:13.019710 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5c8cc3-c4d6-454c-abec-94ab44f1fdd2 admin admin] PATH_INFO: `/v3/projects/29b4622415af4edfaedfec8dec05ac3e` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:13.020863 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5c8cc3-c4d6-454c-abec-94ab44f1fdd2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=29b4622415af4edfaedfec8dec05ac3e)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:13.022848 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5c8cc3-c4d6-454c-abec-94ab44f1fdd2 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:13.127224 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:42:13.129247 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cf8b96-14c4-4f54-bfb1-c87da8ef9d9f admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:13.132574 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-61cf8b96-14c4-4f54-bfb1-c87da8ef9d9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:42:13.136338 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-7b5c8cc3-c4d6-454c-abec-94ab44f1fdd2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29b4622415af4edfaedfec8dec05ac3e', 'request_id': 'req-7b5c8cc3-c4d6-454c-abec-94ab44f1fdd2'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:13.152056 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cf8b96-14c4-4f54-bfb1-c87da8ef9d9f admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:13.156554 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-7b5c8cc3-c4d6-454c-abec-94ab44f1fdd2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29b4622415af4edfaedfec8dec05ac3e', 'request_id': 'req-7b5c8cc3-c4d6-454c-abec-94ab44f1fdd2'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:13.202057 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:42:13.214428 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f987902b-c226-444c-89c7-95f8ff73b51c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:13.215051 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f987902b-c226-444c-89c7-95f8ff73b51c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:13.215328 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f987902b-c226-444c-89c7-95f8ff73b51c admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:13.216772 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f987902b-c226-444c-89c7-95f8ff73b51c admin admin] PATH_INFO: `/v3/projects/50f38633e8454957ab1867de4ceab7ab` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:13.216772 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f987902b-c226-444c-89c7-95f8ff73b51c admin admin] RBAC: Authorizing `identity:delete_project(project_id=50f38633e8454957ab1867de4ceab7ab)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:13.218692 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f987902b-c226-444c-89c7-95f8ff73b51c admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:13.236887 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-7b5c8cc3-c4d6-454c-abec-94ab44f1fdd2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29b4622415af4edfaedfec8dec05ac3e', 'request_id': 'req-7b5c8cc3-c4d6-454c-abec-94ab44f1fdd2'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:13.250231 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-7b5c8cc3-c4d6-454c-abec-94ab44f1fdd2 admin admin] The token cache is being invalidate because project 29b4622415af4edfaedfec8dec05ac3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 08:42:13.250520 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-7b5c8cc3-c4d6-454c-abec-94ab44f1fdd2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:13.251447 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-7b5c8cc3-c4d6-454c-abec-94ab44f1fdd2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:13.253369 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 898/1786] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 08:42:12 2022] DELETE /identity/v3/projects/29b4622415af4edfaedfec8dec05ac3e => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:42:13.257849 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3af3e73-eb25-47d2-a48c-f10db82ebe38 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:13.258135 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3af3e73-eb25-47d2-a48c-f10db82ebe38 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:13.258408 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3af3e73-eb25-47d2-a48c-f10db82ebe38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:13.280001 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-b3af3e73-eb25-47d2-a48c-f10db82ebe38 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:42:13.281252 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-b3af3e73-eb25-47d2-a48c-f10db82ebe38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:42:13.291167 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 899/1787] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:42:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 08:42:13.299170 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddba553-8589-49f8-bf05-eae1a345a1a0 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:13.299484 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddba553-8589-49f8-bf05-eae1a345a1a0 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:13.299875 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddba553-8589-49f8-bf05-eae1a345a1a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:13.318601 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-2ddba553-8589-49f8-bf05-eae1a345a1a0 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:42:13.319773 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-2ddba553-8589-49f8-bf05-eae1a345a1a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:42:13.327716 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 900/1788] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:42:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 08:42:13.332288 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddba553-8589-49f8-bf05-eae1a345a1a0 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:13.341269 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddba553-8589-49f8-bf05-eae1a345a1a0 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:13.352709 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-f987902b-c226-444c-89c7-95f8ff73b51c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50f38633e8454957ab1867de4ceab7ab', 'request_id': 'req-f987902b-c226-444c-89c7-95f8ff73b51c'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:13.398826 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b83376-5d7c-4bb2-a7e8-352b60d0e0a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:13.399441 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b83376-5d7c-4bb2-a7e8-352b60d0e0a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:13.399775 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b83376-5d7c-4bb2-a7e8-352b60d0e0a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:13.399983 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b83376-5d7c-4bb2-a7e8-352b60d0e0a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:13.400652 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b83376-5d7c-4bb2-a7e8-352b60d0e0a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:42:13.400985 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b83376-5d7c-4bb2-a7e8-352b60d0e0a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:13.403067 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b83376-5d7c-4bb2-a7e8-352b60d0e0a4 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:13.414465 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 901/1789] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Aug 10 08:42:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:42:13.418792 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b83376-5d7c-4bb2-a7e8-352b60d0e0a4 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:13.419354 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-f987902b-c226-444c-89c7-95f8ff73b51c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50f38633e8454957ab1867de4ceab7ab', 'request_id': 'req-f987902b-c226-444c-89c7-95f8ff73b51c'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:13.427336 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-f987902b-c226-444c-89c7-95f8ff73b51c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50f38633e8454957ab1867de4ceab7ab', 'request_id': 'req-f987902b-c226-444c-89c7-95f8ff73b51c'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:13.429519 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b83376-5d7c-4bb2-a7e8-352b60d0e0a4 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:13.434462 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636d6ceb-d895-446d-b11a-a5f66a264557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:13.435174 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636d6ceb-d895-446d-b11a-a5f66a264557 admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:13.435451 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636d6ceb-d895-446d-b11a-a5f66a264557 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:13.435721 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636d6ceb-d895-446d-b11a-a5f66a264557 admin admin] PATH_INFO: `/v3/projects` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:13.436335 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636d6ceb-d895-446d-b11a-a5f66a264557 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:13.439141 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636d6ceb-d895-446d-b11a-a5f66a264557 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:13.439357 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-f987902b-c226-444c-89c7-95f8ff73b51c admin admin] The token cache is being invalidate because project 50f38633e8454957ab1867de4ceab7ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 08:42:13.439647 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-f987902b-c226-444c-89c7-95f8ff73b51c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:13.440331 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-f987902b-c226-444c-89c7-95f8ff73b51c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:13.445981 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 889/1790] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 08:42:13 2022] DELETE /identity/v3/projects/50f38633e8454957ab1867de4ceab7ab => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 08:42:13.447364 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37da583e-e66d-4b1d-b466-56171440d116 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:13.447652 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37da583e-e66d-4b1d-b466-56171440d116 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:13.447927 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37da583e-e66d-4b1d-b466-56171440d116 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:13.470205 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-37da583e-e66d-4b1d-b466-56171440d116 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:42:13.471066 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-37da583e-e66d-4b1d-b466-56171440d116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:42:13.480940 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 890/1791] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:42:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:42:13.489915 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e4d53c-6bdb-4b03-b88f-eded031a0dd1 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:13.501133 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e4d53c-6bdb-4b03-b88f-eded031a0dd1 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:13.501133 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e4d53c-6bdb-4b03-b88f-eded031a0dd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:13.512910 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-69e4d53c-6bdb-4b03-b88f-eded031a0dd1 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:42:13.514194 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-69e4d53c-6bdb-4b03-b88f-eded031a0dd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:42:13.523120 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 891/1792] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:42:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:42:13.528686 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e4d53c-6bdb-4b03-b88f-eded031a0dd1 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:13.539591 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e4d53c-6bdb-4b03-b88f-eded031a0dd1 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:13.545121 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56860c25-61e7-479b-acae-f36d405bdf51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:13.545790 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56860c25-61e7-479b-acae-f36d405bdf51 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:13.546538 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56860c25-61e7-479b-acae-f36d405bdf51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:13.546538 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56860c25-61e7-479b-acae-f36d405bdf51 admin admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:13.550660 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56860c25-61e7-479b-acae-f36d405bdf51 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:42:13.550660 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56860c25-61e7-479b-acae-f36d405bdf51 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:13.550660 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56860c25-61e7-479b-acae-f36d405bdf51 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:13.562906 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 892/1793] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Aug 10 08:42:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:42:13.567302 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56860c25-61e7-479b-acae-f36d405bdf51 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:13.578231 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56860c25-61e7-479b-acae-f36d405bdf51 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:13.579228 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 902/1794] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 08:42:13 2022] POST /identity/v3/projects => generated 365 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:42:13.583399 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636d6ceb-d895-446d-b11a-a5f66a264557 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:13.585040 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-636d6ceb-d895-446d-b11a-a5f66a264557 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:42:13.639464 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61821eea-64bc-4709-8296-1ada6c3ce2b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:13.640177 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61821eea-64bc-4709-8296-1ada6c3ce2b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:13.640528 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61821eea-64bc-4709-8296-1ada6c3ce2b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:13.640893 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61821eea-64bc-4709-8296-1ada6c3ce2b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:13.641591 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61821eea-64bc-4709-8296-1ada6c3ce2b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:13.643535 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61821eea-64bc-4709-8296-1ada6c3ce2b2 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:13.655300 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636d6ceb-d895-446d-b11a-a5f66a264557 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:13.662273 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971228ca-fbdd-4def-b545-0372f10e3f55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:13.662889 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971228ca-fbdd-4def-b545-0372f10e3f55 admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:13.663167 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971228ca-fbdd-4def-b545-0372f10e3f55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:13.663479 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971228ca-fbdd-4def-b545-0372f10e3f55 admin admin] PATH_INFO: `/v3/users` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:13.664107 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971228ca-fbdd-4def-b545-0372f10e3f55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:13.666378 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971228ca-fbdd-4def-b545-0372f10e3f55 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:13.730534 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 893/1795] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 08:42:13 2022] POST /identity/v3/projects => generated 365 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:42:13.734623 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61821eea-64bc-4709-8296-1ada6c3ce2b2 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:13.779582 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 903/1796] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 08:42:13 2022] POST /identity/v3/users => generated 283 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:42:13.783736 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971228ca-fbdd-4def-b545-0372f10e3f55 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:13.791233 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:42:13.793624 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971228ca-fbdd-4def-b545-0372f10e3f55 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:13.798457 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7554c9e4-c63e-4374-84b6-7c51f2307f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:13.799060 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7554c9e4-c63e-4374-84b6-7c51f2307f91 admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:13.799341 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7554c9e4-c63e-4374-84b6-7c51f2307f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:13.799850 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7554c9e4-c63e-4374-84b6-7c51f2307f91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:13.800025 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61821eea-64bc-4709-8296-1ada6c3ce2b2 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:13.800321 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7554c9e4-c63e-4374-84b6-7c51f2307f91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:13.801982 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7554c9e4-c63e-4374-84b6-7c51f2307f91 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:13.804339 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b0067e-f205-4515-9966-3e41a8ab5f51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:13.804966 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b0067e-f205-4515-9966-3e41a8ab5f51 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:13.805251 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b0067e-f205-4515-9966-3e41a8ab5f51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:13.805522 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b0067e-f205-4515-9966-3e41a8ab5f51 admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:13.806143 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b0067e-f205-4515-9966-3e41a8ab5f51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:13.807404 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:DELETE=5,keystone:SELECT=90,keystone:INSERT=5 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:42:13.808098 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b0067e-f205-4515-9966-3e41a8ab5f51 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:13.812514 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 904/1797] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:42:13 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 08:42:13.817045 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7554c9e4-c63e-4374-84b6-7c51f2307f91 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:13.826818 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7554c9e4-c63e-4374-84b6-7c51f2307f91 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:13.832236 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b287502-f3ad-472e-b5cc-a856886e3f38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:13.832822 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b287502-f3ad-472e-b5cc-a856886e3f38 admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:13.833140 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b287502-f3ad-472e-b5cc-a856886e3f38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:13.833462 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b287502-f3ad-472e-b5cc-a856886e3f38 admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:13.834040 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b287502-f3ad-472e-b5cc-a856886e3f38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:13.835663 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b287502-f3ad-472e-b5cc-a856886e3f38 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:13.846008 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 905/1798] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:42:13 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 08:42:13.849945 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b287502-f3ad-472e-b5cc-a856886e3f38 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:13.863571 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b287502-f3ad-472e-b5cc-a856886e3f38 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:13.865158 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce93de52-d552-4547-905f-4f7f8438fd0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:13.865802 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce93de52-d552-4547-905f-4f7f8438fd0e admin admin] REQUEST_METHOD: `PUT` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:13.866106 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce93de52-d552-4547-905f-4f7f8438fd0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:13.866365 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce93de52-d552-4547-905f-4f7f8438fd0e admin admin] PATH_INFO: `/v3/projects/a14cbb13a8174f0fb62ffe8a1f0ef00f/users/0bada08693a549749f98680868f6ada1/roles/44b64db484f24a46bf0b7ced46b519bf` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:13.883919 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce93de52-d552-4547-905f-4f7f8438fd0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a14cbb13a8174f0fb62ffe8a1f0ef00f, user_id=0bada08693a549749f98680868f6ada1, role_id=44b64db484f24a46bf0b7ced46b519bf)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:13.888428 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce93de52-d552-4547-905f-4f7f8438fd0e admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:13.913208 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 894/1799] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 08:42:13 2022] POST /identity/v3/users => generated 283 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 08:42:13.918050 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b0067e-f205-4515-9966-3e41a8ab5f51 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:13.933372 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b0067e-f205-4515-9966-3e41a8ab5f51 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:13.940349 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d880f2c-5f3e-4c05-b30e-98a2babd6fa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:13.943674 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d880f2c-5f3e-4c05-b30e-98a2babd6fa4 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:13.943674 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d880f2c-5f3e-4c05-b30e-98a2babd6fa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:13.943674 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d880f2c-5f3e-4c05-b30e-98a2babd6fa4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:13.943674 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d880f2c-5f3e-4c05-b30e-98a2babd6fa4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:13.944025 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d880f2c-5f3e-4c05-b30e-98a2babd6fa4 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:13.957650 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 895/1800] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:42:13 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:42:13.961639 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d880f2c-5f3e-4c05-b30e-98a2babd6fa4 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:13.972523 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d880f2c-5f3e-4c05-b30e-98a2babd6fa4 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:13.978038 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7b8741-85bd-470a-a5f9-44dbd30b1f14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:13.978038 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7b8741-85bd-470a-a5f9-44dbd30b1f14 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:13.978038 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7b8741-85bd-470a-a5f9-44dbd30b1f14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:13.978537 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7b8741-85bd-470a-a5f9-44dbd30b1f14 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:13.979322 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7b8741-85bd-470a-a5f9-44dbd30b1f14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:13.980924 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7b8741-85bd-470a-a5f9-44dbd30b1f14 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:13.992346 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 896/1801] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:42:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:42:13.996458 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7b8741-85bd-470a-a5f9-44dbd30b1f14 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:14.006060 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 906/1802] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 08:42:13 2022] PUT /identity/v3/projects/a14cbb13a8174f0fb62ffe8a1f0ef00f/users/0bada08693a549749f98680868f6ada1/roles/44b64db484f24a46bf0b7ced46b519bf => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 08:42:14.008739 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7b8741-85bd-470a-a5f9-44dbd30b1f14 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:14.018289 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a31a01d-5f1c-488c-a45f-460c2e890ce6 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:14.018571 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a31a01d-5f1c-488c-a45f-460c2e890ce6 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:14.018844 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a31a01d-5f1c-488c-a45f-460c2e890ce6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:14.050077 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-6a31a01d-5f1c-488c-a45f-460c2e890ce6 None None] MFA Rules not processed for user `0bada08693a549749f98680868f6ada1`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:42:14.051470 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-6a31a01d-5f1c-488c-a45f-460c2e890ce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:42:14.083997 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7d4b3c-3207-4b19-bd6f-e44e3bbf7735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:14.084625 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7d4b3c-3207-4b19-bd6f-e44e3bbf7735 admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:14.084908 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7d4b3c-3207-4b19-bd6f-e44e3bbf7735 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:14.085511 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7d4b3c-3207-4b19-bd6f-e44e3bbf7735 admin admin] PATH_INFO: `/v3/projects/f642d493ee644770882a093284c1dcdb/users/ee09c0be6cd34045a2ebf7cb537fdc19/roles/44b64db484f24a46bf0b7ced46b519bf` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:14.107015 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7d4b3c-3207-4b19-bd6f-e44e3bbf7735 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f642d493ee644770882a093284c1dcdb, user_id=ee09c0be6cd34045a2ebf7cb537fdc19, role_id=44b64db484f24a46bf0b7ced46b519bf)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:14.109866 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7d4b3c-3207-4b19-bd6f-e44e3bbf7735 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:14.134031 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 907/1803] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:42:14 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:42:14.135117 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a31a01d-5f1c-488c-a45f-460c2e890ce6 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:14.148794 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a31a01d-5f1c-488c-a45f-460c2e890ce6 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:14.154515 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16eed10-47ee-49b2-8f65-006eeb814079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:14.155274 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16eed10-47ee-49b2-8f65-006eeb814079 admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:14.155388 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16eed10-47ee-49b2-8f65-006eeb814079 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:14.155822 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16eed10-47ee-49b2-8f65-006eeb814079 admin admin] PATH_INFO: `/v3/projects` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:14.156342 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16eed10-47ee-49b2-8f65-006eeb814079 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:14.159373 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16eed10-47ee-49b2-8f65-006eeb814079 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:14.164000 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 897/1804] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 08:42:13 2022] PUT /identity/v3/projects/f642d493ee644770882a093284c1dcdb/users/ee09c0be6cd34045a2ebf7cb537fdc19/roles/44b64db484f24a46bf0b7ced46b519bf => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 08:42:14.177384 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80e1a0d-e18e-4758-9f76-91dc33bca26a None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:14.177864 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80e1a0d-e18e-4758-9f76-91dc33bca26a None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:14.178085 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80e1a0d-e18e-4758-9f76-91dc33bca26a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:14.230250 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-c80e1a0d-e18e-4758-9f76-91dc33bca26a None None] MFA Rules not processed for user `ee09c0be6cd34045a2ebf7cb537fdc19`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:42:14.232370 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-c80e1a0d-e18e-4758-9f76-91dc33bca26a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:42:14.264242 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 908/1805] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 08:42:14 2022] POST /identity/v3/projects => generated 365 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:42:14.268626 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16eed10-47ee-49b2-8f65-006eeb814079 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:14.317403 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 898/1806] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:42:14 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 08:42:14.321574 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80e1a0d-e18e-4758-9f76-91dc33bca26a None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:14.352039 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:42:14.359887 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16eed10-47ee-49b2-8f65-006eeb814079 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:14.366166 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfc690a-27c2-416d-97ae-cad6654b2d07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:14.367201 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfc690a-27c2-416d-97ae-cad6654b2d07 admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:14.367729 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfc690a-27c2-416d-97ae-cad6654b2d07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:14.368085 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfc690a-27c2-416d-97ae-cad6654b2d07 admin admin] PATH_INFO: `/v3/users` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:14.368841 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfc690a-27c2-416d-97ae-cad6654b2d07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:14.371249 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfc690a-27c2-416d-97ae-cad6654b2d07 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:14.413934 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80e1a0d-e18e-4758-9f76-91dc33bca26a None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:14.419793 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb21435-216b-486f-afbc-65bf40e02a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:14.420457 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb21435-216b-486f-afbc-65bf40e02a4d admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:14.420748 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb21435-216b-486f-afbc-65bf40e02a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:14.421287 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb21435-216b-486f-afbc-65bf40e02a4d admin admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:14.421922 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb21435-216b-486f-afbc-65bf40e02a4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:14.425490 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb21435-216b-486f-afbc-65bf40e02a4d admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:14.463125 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 909/1807] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 08:42:14 2022] POST /identity/v3/users => generated 289 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:42:14.467050 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfc690a-27c2-416d-97ae-cad6654b2d07 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:14.480876 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfc690a-27c2-416d-97ae-cad6654b2d07 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:14.485724 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7dcf53-03fd-4549-9020-0f16ad683f46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:14.486359 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7dcf53-03fd-4549-9020-0f16ad683f46 admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:14.486646 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7dcf53-03fd-4549-9020-0f16ad683f46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:14.486914 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7dcf53-03fd-4549-9020-0f16ad683f46 admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:14.487594 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7dcf53-03fd-4549-9020-0f16ad683f46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:14.489237 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7dcf53-03fd-4549-9020-0f16ad683f46 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:14.510485 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 899/1808] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 08:42:14 2022] POST /identity/v3/projects => generated 363 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:42:14.514610 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb21435-216b-486f-afbc-65bf40e02a4d admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:14.524064 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 910/1809] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:42:14 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 08:42:14.526701 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7dcf53-03fd-4549-9020-0f16ad683f46 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:14.553938 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7dcf53-03fd-4549-9020-0f16ad683f46 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:14.566627 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:42:14.617214 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb21435-216b-486f-afbc-65bf40e02a4d admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:14.624861 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfc7826-a785-465f-a35b-38d0222ffef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:14.625015 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a95f568-14a8-4bbe-b65f-615e39cda0b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:14.625125 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfc7826-a785-465f-a35b-38d0222ffef4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:14.625210 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a95f568-14a8-4bbe-b65f-615e39cda0b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:14.625306 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfc7826-a785-465f-a35b-38d0222ffef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:14.625401 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a95f568-14a8-4bbe-b65f-615e39cda0b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:14.625495 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfc7826-a785-465f-a35b-38d0222ffef4 admin admin] PATH_INFO: `/v3/projects/058f48df4756449eb52a520960593e09/users/48e76241b6684e85a3737b454b07ec3b/roles/ec3751692d9c4ecf81b4a71c936c46be` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:14.625583 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a95f568-14a8-4bbe-b65f-615e39cda0b7 admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:14.626067 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a95f568-14a8-4bbe-b65f-615e39cda0b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:14.628092 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a95f568-14a8-4bbe-b65f-615e39cda0b7 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:14.642887 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfc7826-a785-465f-a35b-38d0222ffef4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=058f48df4756449eb52a520960593e09, user_id=48e76241b6684e85a3737b454b07ec3b, role_id=ec3751692d9c4ecf81b4a71c936c46be)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:14.647893 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfc7826-a785-465f-a35b-38d0222ffef4 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:14.728713 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 900/1810] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 08:42:14 2022] POST /identity/v3/users => generated 288 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:42:14.732631 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a95f568-14a8-4bbe-b65f-615e39cda0b7 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:14.742988 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a95f568-14a8-4bbe-b65f-615e39cda0b7 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:14.747831 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7804e4d-2a92-4fa7-948a-88f7eccdfd19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:14.748668 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7804e4d-2a92-4fa7-948a-88f7eccdfd19 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:14.748972 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7804e4d-2a92-4fa7-948a-88f7eccdfd19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:14.749251 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7804e4d-2a92-4fa7-948a-88f7eccdfd19 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:14.750266 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7804e4d-2a92-4fa7-948a-88f7eccdfd19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:14.752509 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7804e4d-2a92-4fa7-948a-88f7eccdfd19 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:14.760054 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 911/1811] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 08:42:14 2022] PUT /identity/v3/projects/058f48df4756449eb52a520960593e09/users/48e76241b6684e85a3737b454b07ec3b/roles/ec3751692d9c4ecf81b4a71c936c46be => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:42:14.764686 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfc7826-a785-465f-a35b-38d0222ffef4 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:14.766984 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 901/1812] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:42:14 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 08:42:14.773506 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7804e4d-2a92-4fa7-948a-88f7eccdfd19 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:14.784139 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7804e4d-2a92-4fa7-948a-88f7eccdfd19 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:14.841791 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfc7826-a785-465f-a35b-38d0222ffef4 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:14.847140 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2722c287-de16-4dfd-b3b0-1ba6ddbd3ef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:14.847771 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2722c287-de16-4dfd-b3b0-1ba6ddbd3ef1 admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:14.848051 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2722c287-de16-4dfd-b3b0-1ba6ddbd3ef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:14.848326 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2722c287-de16-4dfd-b3b0-1ba6ddbd3ef1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:14.849668 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2722c287-de16-4dfd-b3b0-1ba6ddbd3ef1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:14.851318 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2722c287-de16-4dfd-b3b0-1ba6ddbd3ef1 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:14.862415 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bc0468-cc69-4af5-911b-db797da9643c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:14.863222 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bc0468-cc69-4af5-911b-db797da9643c admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:14.863603 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bc0468-cc69-4af5-911b-db797da9643c admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:14.864113 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 912/1813] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:42:14 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:42:14.864230 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bc0468-cc69-4af5-911b-db797da9643c admin admin] PATH_INFO: `/v3/projects/2b8d070f60504460843fd717135805d8/users/66797a43aa10465a8cb111402474ea96/roles/ec3751692d9c4ecf81b4a71c936c46be` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:14.868207 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2722c287-de16-4dfd-b3b0-1ba6ddbd3ef1 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:14.880028 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2722c287-de16-4dfd-b3b0-1ba6ddbd3ef1 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:14.882926 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bc0468-cc69-4af5-911b-db797da9643c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b8d070f60504460843fd717135805d8, user_id=66797a43aa10465a8cb111402474ea96, role_id=ec3751692d9c4ecf81b4a71c936c46be)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:14.885996 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7244d8d-ba2f-4dbc-a349-93207aba9ae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:14.886278 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bc0468-cc69-4af5-911b-db797da9643c admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:14.898117 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7244d8d-ba2f-4dbc-a349-93207aba9ae0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:14.898117 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7244d8d-ba2f-4dbc-a349-93207aba9ae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:14.898117 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7244d8d-ba2f-4dbc-a349-93207aba9ae0 admin admin] PATH_INFO: `/v3/projects/058f48df4756449eb52a520960593e09/users/48e76241b6684e85a3737b454b07ec3b/roles/ec3751692d9c4ecf81b4a71c936c46be` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:14.898117 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7244d8d-ba2f-4dbc-a349-93207aba9ae0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=058f48df4756449eb52a520960593e09, user_id=48e76241b6684e85a3737b454b07ec3b, role_id=ec3751692d9c4ecf81b4a71c936c46be)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:14.898117 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7244d8d-ba2f-4dbc-a349-93207aba9ae0 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:14.902919 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 913/1814] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 08:42:14 2022] PUT /identity/v3/projects/058f48df4756449eb52a520960593e09/users/48e76241b6684e85a3737b454b07ec3b/roles/ec3751692d9c4ecf81b4a71c936c46be => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 08:42:14.914250 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa471016-1dcc-4d9e-bac7-86369c7bd087 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:14.914563 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa471016-1dcc-4d9e-bac7-86369c7bd087 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:14.919546 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa471016-1dcc-4d9e-bac7-86369c7bd087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:14.946160 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-fa471016-1dcc-4d9e-bac7-86369c7bd087 None None] MFA Rules not processed for user `48e76241b6684e85a3737b454b07ec3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:42:14.948452 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-fa471016-1dcc-4d9e-bac7-86369c7bd087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:42:14.984114 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 902/1815] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 08:42:14 2022] PUT /identity/v3/projects/2b8d070f60504460843fd717135805d8/users/66797a43aa10465a8cb111402474ea96/roles/ec3751692d9c4ecf81b4a71c936c46be => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:42:14.989882 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bc0468-cc69-4af5-911b-db797da9643c admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:14.996980 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:42:15.030435 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 914/1816] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:42:14 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 08:42:15.042461 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa471016-1dcc-4d9e-bac7-86369c7bd087 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:15.043827 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-fa471016-1dcc-4d9e-bac7-86369c7bd087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:42:15.049757 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:42:15.055524 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bc0468-cc69-4af5-911b-db797da9643c admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:15.060983 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fa0661-43ae-4550-9ee8-f76a6296c966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:15.061541 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fa0661-43ae-4550-9ee8-f76a6296c966 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:15.061809 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fa0661-43ae-4550-9ee8-f76a6296c966 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:15.062029 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fa0661-43ae-4550-9ee8-f76a6296c966 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:15.063140 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fa0661-43ae-4550-9ee8-f76a6296c966 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:15.065112 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fa0661-43ae-4550-9ee8-f76a6296c966 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:15.075692 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 903/1817] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:42:14 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:42:15.080523 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fa0661-43ae-4550-9ee8-f76a6296c966 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:15.092244 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fa0661-43ae-4550-9ee8-f76a6296c966 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:15.097127 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48421ee-49d4-4084-bc2e-be68e32a6ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:15.097944 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48421ee-49d4-4084-bc2e-be68e32a6ab2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:15.098295 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48421ee-49d4-4084-bc2e-be68e32a6ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:15.098651 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48421ee-49d4-4084-bc2e-be68e32a6ab2 admin admin] PATH_INFO: `/v3/projects/2b8d070f60504460843fd717135805d8/users/66797a43aa10465a8cb111402474ea96/roles/ec3751692d9c4ecf81b4a71c936c46be` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:15.100748 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48421ee-49d4-4084-bc2e-be68e32a6ab2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b8d070f60504460843fd717135805d8, user_id=66797a43aa10465a8cb111402474ea96, role_id=ec3751692d9c4ecf81b4a71c936c46be)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:15.103528 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48421ee-49d4-4084-bc2e-be68e32a6ab2 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:15.116932 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa471016-1dcc-4d9e-bac7-86369c7bd087 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:15.120065 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 904/1818] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 08:42:15 2022] PUT /identity/v3/projects/2b8d070f60504460843fd717135805d8/users/66797a43aa10465a8cb111402474ea96/roles/ec3751692d9c4ecf81b4a71c936c46be => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:42:15.129832 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09adc2d1-33aa-441a-9216-c42b15b3ec4e None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:15.130449 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09adc2d1-33aa-441a-9216-c42b15b3ec4e None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:15.130717 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09adc2d1-33aa-441a-9216-c42b15b3ec4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:15.163500 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-09adc2d1-33aa-441a-9216-c42b15b3ec4e None None] MFA Rules not processed for user `66797a43aa10465a8cb111402474ea96`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:42:15.164793 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-09adc2d1-33aa-441a-9216-c42b15b3ec4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:42:15.170260 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7684d8e8-b4fc-4367-a031-78528f7d8930 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ba74e28ced4cbba857f54222dfbe41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:15.170965 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7684d8e8-b4fc-4367-a031-78528f7d8930 service cinder] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:15.171314 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7684d8e8-b4fc-4367-a031-78528f7d8930 service cinder] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:15.171659 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7684d8e8-b4fc-4367-a031-78528f7d8930 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:15.255132 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 905/1819] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:42:15 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:42:15.261837 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7684d8e8-b4fc-4367-a031-78528f7d8930 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:15.263515 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7684d8e8-b4fc-4367-a031-78528f7d8930 service cinder] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:15.269177 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09adc2d1-33aa-441a-9216-c42b15b3ec4e None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:15.271972 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-09adc2d1-33aa-441a-9216-c42b15b3ec4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:42:15.275254 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 915/1820] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:42:15 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:42:15.285010 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09adc2d1-33aa-441a-9216-c42b15b3ec4e None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:15.289493 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58b51d8-5c3f-4e63-bdac-63a283b073b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ba74e28ced4cbba857f54222dfbe41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:15.291998 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58b51d8-5c3f-4e63-bdac-63a283b073b8 service cinder] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:15.291998 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58b51d8-5c3f-4e63-bdac-63a283b073b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:15.291998 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58b51d8-5c3f-4e63-bdac-63a283b073b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:15.303894 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58b51d8-5c3f-4e63-bdac-63a283b073b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:15.305472 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58b51d8-5c3f-4e63-bdac-63a283b073b8 service cinder] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:15.316624 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 906/1821] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:42:15 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:42:15.519913 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7684d8e8-b4fc-4367-a031-78528f7d8930 service cinder] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:15.526333 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58b51d8-5c3f-4e63-bdac-63a283b073b8 service cinder] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:15.533072 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7684d8e8-b4fc-4367-a031-78528f7d8930 service cinder] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:15.533527 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4212281-dbac-4cdf-b2c8-96feca69bde3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ba74e28ced4cbba857f54222dfbe41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:15.534134 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4212281-dbac-4cdf-b2c8-96feca69bde3 service cinder] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:15.534415 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4212281-dbac-4cdf-b2c8-96feca69bde3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:15.534684 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4212281-dbac-4cdf-b2c8-96feca69bde3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:15.535577 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58b51d8-5c3f-4e63-bdac-63a283b073b8 service cinder] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:15.539651 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6e1ef0-4daf-4524-9834-037420991cff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ba74e28ced4cbba857f54222dfbe41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:15.540777 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6e1ef0-4daf-4524-9834-037420991cff service cinder] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:15.541095 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6e1ef0-4daf-4524-9834-037420991cff service cinder] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:15.541373 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6e1ef0-4daf-4524-9834-037420991cff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:15.593001 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4212281-dbac-4cdf-b2c8-96feca69bde3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:15.594440 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4212281-dbac-4cdf-b2c8-96feca69bde3 service cinder] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:15.603229 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6e1ef0-4daf-4524-9834-037420991cff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:15.604080 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 916/1822] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:42:15 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:42:15.604875 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6e1ef0-4daf-4524-9834-037420991cff service cinder] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:15.608745 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:42:15.618122 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 907/1823] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:42:15 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 08:42:19.309140 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4212281-dbac-4cdf-b2c8-96feca69bde3 service cinder] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:19.319439 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4212281-dbac-4cdf-b2c8-96feca69bde3 service cinder] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:19.325591 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f55a2c1-4f91-4869-bdd9-b3253a03e8d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ba74e28ced4cbba857f54222dfbe41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:19.326228 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f55a2c1-4f91-4869-bdd9-b3253a03e8d2 service cinder] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:19.326510 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f55a2c1-4f91-4869-bdd9-b3253a03e8d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:19.330500 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f55a2c1-4f91-4869-bdd9-b3253a03e8d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:19.330500 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-9f55a2c1-4f91-4869-bdd9-b3253a03e8d2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:42:19.334688 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:42:19.397356 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f55a2c1-4f91-4869-bdd9-b3253a03e8d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:19.398834 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f55a2c1-4f91-4869-bdd9-b3253a03e8d2 service cinder] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:19.409249 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 917/1824] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:42:19 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:42:21.952462 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6e1ef0-4daf-4524-9834-037420991cff service cinder] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:22.031570 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6e1ef0-4daf-4524-9834-037420991cff service cinder] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:22.036376 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7d82bb-4893-4296-81dd-3599ad930cc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:22.037055 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7d82bb-4893-4296-81dd-3599ad930cc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:22.037332 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7d82bb-4893-4296-81dd-3599ad930cc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:22.037654 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7d82bb-4893-4296-81dd-3599ad930cc1 admin admin] PATH_INFO: `/v3/users/ee09c0be6cd34045a2ebf7cb537fdc19` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:22.038780 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7d82bb-4893-4296-81dd-3599ad930cc1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee09c0be6cd34045a2ebf7cb537fdc19)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:22.040716 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7d82bb-4893-4296-81dd-3599ad930cc1 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:22.216220 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-fa7d82bb-4893-4296-81dd-3599ad930cc1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee09c0be6cd34045a2ebf7cb537fdc19'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:22.231223 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-fa7d82bb-4893-4296-81dd-3599ad930cc1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee09c0be6cd34045a2ebf7cb537fdc19'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:22.237970 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-fa7d82bb-4893-4296-81dd-3599ad930cc1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee09c0be6cd34045a2ebf7cb537fdc19'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:22.238706 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-fa7d82bb-4893-4296-81dd-3599ad930cc1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee09c0be6cd34045a2ebf7cb537fdc19'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:22.239316 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-fa7d82bb-4893-4296-81dd-3599ad930cc1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee09c0be6cd34045a2ebf7cb537fdc19'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:22.312542 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 908/1825] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 08:42:21 2022] DELETE /identity/v3/users/ee09c0be6cd34045a2ebf7cb537fdc19 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:42:22.318376 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734409c6-05af-4407-bcda-96b476578b8d None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:22.318841 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734409c6-05af-4407-bcda-96b476578b8d None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:22.318933 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734409c6-05af-4407-bcda-96b476578b8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:22.343067 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-734409c6-05af-4407-bcda-96b476578b8d None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:42:22.344972 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-734409c6-05af-4407-bcda-96b476578b8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:42:22.407653 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 918/1826] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:42:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:42:22.419041 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7d82bb-4893-4296-81dd-3599ad930cc1 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:22.420539 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-fa7d82bb-4893-4296-81dd-3599ad930cc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:42:22.498973 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7d82bb-4893-4296-81dd-3599ad930cc1 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:22.503265 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e17ba8e-5df0-4fb0-b29e-bc5bd844d6cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:22.503883 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e17ba8e-5df0-4fb0-b29e-bc5bd844d6cb service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:22.504228 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e17ba8e-5df0-4fb0-b29e-bc5bd844d6cb service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:22.504529 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e17ba8e-5df0-4fb0-b29e-bc5bd844d6cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:22.515154 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e17ba8e-5df0-4fb0-b29e-bc5bd844d6cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:22.517265 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e17ba8e-5df0-4fb0-b29e-bc5bd844d6cb service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:22.527523 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 909/1827] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:42:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:42:23.032605 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734409c6-05af-4407-bcda-96b476578b8d None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:23.034224 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-734409c6-05af-4407-bcda-96b476578b8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:42:23.046292 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734409c6-05af-4407-bcda-96b476578b8d None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:23.051679 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3887724-3234-4d7a-90dc-193dec73bd7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:23.052396 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3887724-3234-4d7a-90dc-193dec73bd7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:23.052599 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3887724-3234-4d7a-90dc-193dec73bd7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:23.052890 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3887724-3234-4d7a-90dc-193dec73bd7c admin admin] PATH_INFO: `/v3/projects/f642d493ee644770882a093284c1dcdb` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:23.054044 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3887724-3234-4d7a-90dc-193dec73bd7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f642d493ee644770882a093284c1dcdb)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:23.055977 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3887724-3234-4d7a-90dc-193dec73bd7c admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:23.186051 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-b3887724-3234-4d7a-90dc-193dec73bd7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f642d493ee644770882a093284c1dcdb', 'request_id': 'req-b3887724-3234-4d7a-90dc-193dec73bd7c'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:23.191699 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-b3887724-3234-4d7a-90dc-193dec73bd7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f642d493ee644770882a093284c1dcdb', 'request_id': 'req-b3887724-3234-4d7a-90dc-193dec73bd7c'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:23.270643 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-b3887724-3234-4d7a-90dc-193dec73bd7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f642d493ee644770882a093284c1dcdb', 'request_id': 'req-b3887724-3234-4d7a-90dc-193dec73bd7c'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:23.283314 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-b3887724-3234-4d7a-90dc-193dec73bd7c admin admin] The token cache is being invalidate because project f642d493ee644770882a093284c1dcdb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 08:42:23.283535 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-b3887724-3234-4d7a-90dc-193dec73bd7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:23.284163 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-b3887724-3234-4d7a-90dc-193dec73bd7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:23.285725 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 919/1828] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 08:42:23 2022] DELETE /identity/v3/projects/f642d493ee644770882a093284c1dcdb => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:42:23.289871 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e17ba8e-5df0-4fb0-b29e-bc5bd844d6cb service neutron] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:23.299512 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e17ba8e-5df0-4fb0-b29e-bc5bd844d6cb service neutron] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:23.315896 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:42:23.360510 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e963c4f-63b8-4225-ae4b-97c2d27f90b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:23.361169 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e963c4f-63b8-4225-ae4b-97c2d27f90b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:23.361447 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e963c4f-63b8-4225-ae4b-97c2d27f90b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:23.361714 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e963c4f-63b8-4225-ae4b-97c2d27f90b0 admin admin] PATH_INFO: `/v3/users/66797a43aa10465a8cb111402474ea96` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:23.362865 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e963c4f-63b8-4225-ae4b-97c2d27f90b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=66797a43aa10465a8cb111402474ea96)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:23.364774 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e963c4f-63b8-4225-ae4b-97c2d27f90b0 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:23.536406 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-9e963c4f-63b8-4225-ae4b-97c2d27f90b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66797a43aa10465a8cb111402474ea96'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:23.548495 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-9e963c4f-63b8-4225-ae4b-97c2d27f90b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66797a43aa10465a8cb111402474ea96'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:23.555648 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-9e963c4f-63b8-4225-ae4b-97c2d27f90b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66797a43aa10465a8cb111402474ea96'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:23.556339 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-9e963c4f-63b8-4225-ae4b-97c2d27f90b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66797a43aa10465a8cb111402474ea96'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:23.556966 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-9e963c4f-63b8-4225-ae4b-97c2d27f90b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66797a43aa10465a8cb111402474ea96'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:23.658987 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 910/1829] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 08:42:23 2022] DELETE /identity/v3/users/66797a43aa10465a8cb111402474ea96 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:42:24.084161 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3887724-3234-4d7a-90dc-193dec73bd7c admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:24.085573 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-b3887724-3234-4d7a-90dc-193dec73bd7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:42:24.099029 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3887724-3234-4d7a-90dc-193dec73bd7c admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:24.130989 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:42:24.171717 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82afbc6-894c-4f56-bbef-2f9c491ac49a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:24.173076 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82afbc6-894c-4f56-bbef-2f9c491ac49a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:24.173369 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82afbc6-894c-4f56-bbef-2f9c491ac49a admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:24.173634 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82afbc6-894c-4f56-bbef-2f9c491ac49a admin admin] PATH_INFO: `/v3/projects/2b8d070f60504460843fd717135805d8` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:24.174873 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82afbc6-894c-4f56-bbef-2f9c491ac49a admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b8d070f60504460843fd717135805d8)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:24.176955 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82afbc6-894c-4f56-bbef-2f9c491ac49a admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:24.265680 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-d82afbc6-894c-4f56-bbef-2f9c491ac49a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b8d070f60504460843fd717135805d8', 'request_id': 'req-d82afbc6-894c-4f56-bbef-2f9c491ac49a'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:24.272594 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-d82afbc6-894c-4f56-bbef-2f9c491ac49a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b8d070f60504460843fd717135805d8', 'request_id': 'req-d82afbc6-894c-4f56-bbef-2f9c491ac49a'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:24.345130 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-d82afbc6-894c-4f56-bbef-2f9c491ac49a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b8d070f60504460843fd717135805d8', 'request_id': 'req-d82afbc6-894c-4f56-bbef-2f9c491ac49a'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:24.357256 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-d82afbc6-894c-4f56-bbef-2f9c491ac49a admin admin] The token cache is being invalidate because project 2b8d070f60504460843fd717135805d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 08:42:24.357485 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-d82afbc6-894c-4f56-bbef-2f9c491ac49a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:24.358136 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-d82afbc6-894c-4f56-bbef-2f9c491ac49a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:24.360373 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 920/1830] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 08:42:24 2022] DELETE /identity/v3/projects/2b8d070f60504460843fd717135805d8 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:42:24.365170 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3638249-7ea2-48bf-b4af-b3baec2f5331 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:24.369093 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3638249-7ea2-48bf-b4af-b3baec2f5331 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:24.369093 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3638249-7ea2-48bf-b4af-b3baec2f5331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:24.386531 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-f3638249-7ea2-48bf-b4af-b3baec2f5331 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:42:24.387640 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-f3638249-7ea2-48bf-b4af-b3baec2f5331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:42:24.447425 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 911/1831] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:42:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:42:24.456681 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e716b99-84fc-4277-812e-27222bcd2826 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:24.457002 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e716b99-84fc-4277-812e-27222bcd2826 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:24.457386 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e716b99-84fc-4277-812e-27222bcd2826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:24.478832 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-4e716b99-84fc-4277-812e-27222bcd2826 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:42:24.480018 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-4e716b99-84fc-4277-812e-27222bcd2826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:42:24.488700 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 921/1832] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:42:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:42:24.493181 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3638249-7ea2-48bf-b4af-b3baec2f5331 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:24.503147 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3638249-7ea2-48bf-b4af-b3baec2f5331 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:24.507783 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02558682-6348-48a0-bbd1-be3d8b51a3b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:24.509540 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02558682-6348-48a0-bbd1-be3d8b51a3b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:24.509540 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02558682-6348-48a0-bbd1-be3d8b51a3b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:24.509540 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02558682-6348-48a0-bbd1-be3d8b51a3b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:24.509540 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02558682-6348-48a0-bbd1-be3d8b51a3b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:42:24.509954 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02558682-6348-48a0-bbd1-be3d8b51a3b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:24.511416 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02558682-6348-48a0-bbd1-be3d8b51a3b0 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:24.523416 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 912/1833] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Aug 10 08:42:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:42:24.527858 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e716b99-84fc-4277-812e-27222bcd2826 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:24.538002 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e716b99-84fc-4277-812e-27222bcd2826 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:24.542581 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d181e12-d783-4c5e-9330-55e718c9fc1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:24.543205 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d181e12-d783-4c5e-9330-55e718c9fc1b admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:24.543483 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d181e12-d783-4c5e-9330-55e718c9fc1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:24.543778 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d181e12-d783-4c5e-9330-55e718c9fc1b admin admin] PATH_INFO: `/v3/projects` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:24.544392 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d181e12-d783-4c5e-9330-55e718c9fc1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:24.546309 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d181e12-d783-4c5e-9330-55e718c9fc1b admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:24.660632 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 922/1834] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 08:42:24 2022] POST /identity/v3/projects => generated 381 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 08:42:24.664995 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02558682-6348-48a0-bbd1-be3d8b51a3b0 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:24.676299 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02558682-6348-48a0-bbd1-be3d8b51a3b0 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:24.699942 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:42:24.731330 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55da16fb-8192-4110-aaef-6584dc82b405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:24.731935 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55da16fb-8192-4110-aaef-6584dc82b405 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:24.732405 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55da16fb-8192-4110-aaef-6584dc82b405 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:24.732717 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55da16fb-8192-4110-aaef-6584dc82b405 admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:24.733385 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55da16fb-8192-4110-aaef-6584dc82b405 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:24.736287 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55da16fb-8192-4110-aaef-6584dc82b405 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:24.824122 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 913/1835] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 08:42:24 2022] POST /identity/v3/users => generated 291 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:42:24.828222 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d181e12-d783-4c5e-9330-55e718c9fc1b admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:24.839299 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d181e12-d783-4c5e-9330-55e718c9fc1b admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:24.845302 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf16e9b-7a38-4076-90f3-ad9a13b89a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:24.845908 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf16e9b-7a38-4076-90f3-ad9a13b89a0b admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:24.846185 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf16e9b-7a38-4076-90f3-ad9a13b89a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:24.846455 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf16e9b-7a38-4076-90f3-ad9a13b89a0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:24.847089 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf16e9b-7a38-4076-90f3-ad9a13b89a0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:24.848695 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf16e9b-7a38-4076-90f3-ad9a13b89a0b admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:24.858994 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 923/1836] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:42:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 08:42:24.862894 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55da16fb-8192-4110-aaef-6584dc82b405 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:24.873003 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55da16fb-8192-4110-aaef-6584dc82b405 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:24.878317 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd8c0e0-2cab-4396-bbe5-df5bd49c09c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:24.878939 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd8c0e0-2cab-4396-bbe5-df5bd49c09c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:24.879221 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd8c0e0-2cab-4396-bbe5-df5bd49c09c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:24.879495 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd8c0e0-2cab-4396-bbe5-df5bd49c09c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:24.880171 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd8c0e0-2cab-4396-bbe5-df5bd49c09c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:24.881807 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd8c0e0-2cab-4396-bbe5-df5bd49c09c4 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:24.893160 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 914/1837] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:42:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:42:24.897164 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf16e9b-7a38-4076-90f3-ad9a13b89a0b admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:24.907597 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf16e9b-7a38-4076-90f3-ad9a13b89a0b admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:24.912430 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a1aba8-d924-4773-bfdd-5ff0e0c36b61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:24.913054 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a1aba8-d924-4773-bfdd-5ff0e0c36b61 admin admin] REQUEST_METHOD: `PUT` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:24.913339 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a1aba8-d924-4773-bfdd-5ff0e0c36b61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:24.913608 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a1aba8-d924-4773-bfdd-5ff0e0c36b61 admin admin] PATH_INFO: `/v3/projects/525e60831c22404d99d2ca9a378fd3a9/users/1cad005805034957bbfabbad8e0f547e/roles/44b64db484f24a46bf0b7ced46b519bf` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:24.927831 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a1aba8-d924-4773-bfdd-5ff0e0c36b61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=525e60831c22404d99d2ca9a378fd3a9, user_id=1cad005805034957bbfabbad8e0f547e, role_id=44b64db484f24a46bf0b7ced46b519bf)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:24.930532 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a1aba8-d924-4773-bfdd-5ff0e0c36b61 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:24.979869 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 924/1838] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 08:42:24 2022] PUT /identity/v3/projects/525e60831c22404d99d2ca9a378fd3a9/users/1cad005805034957bbfabbad8e0f547e/roles/44b64db484f24a46bf0b7ced46b519bf => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:42:24.991249 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576e22f2-6a7a-4b95-9eda-ad0156cc3928 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:24.991544 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576e22f2-6a7a-4b95-9eda-ad0156cc3928 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:24.991827 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576e22f2-6a7a-4b95-9eda-ad0156cc3928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:25.028799 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-576e22f2-6a7a-4b95-9eda-ad0156cc3928 None None] MFA Rules not processed for user `1cad005805034957bbfabbad8e0f547e`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:42:25.030572 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-576e22f2-6a7a-4b95-9eda-ad0156cc3928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:42:25.113971 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 915/1839] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:42:24 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:42:25.118548 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a1aba8-d924-4773-bfdd-5ff0e0c36b61 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:25.184995 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:42:25.210564 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a1aba8-d924-4773-bfdd-5ff0e0c36b61 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:25.216215 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd9ad75-3c95-4fff-a4da-6096fe5a13d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:25.216863 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd9ad75-3c95-4fff-a4da-6096fe5a13d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:25.217294 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd9ad75-3c95-4fff-a4da-6096fe5a13d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:25.217602 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd9ad75-3c95-4fff-a4da-6096fe5a13d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:25.218363 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd9ad75-3c95-4fff-a4da-6096fe5a13d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:25.220459 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd9ad75-3c95-4fff-a4da-6096fe5a13d3 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:25.325329 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 925/1840] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 08:42:25 2022] POST /identity/v3/projects => generated 381 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:42:25.329671 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576e22f2-6a7a-4b95-9eda-ad0156cc3928 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:25.341280 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576e22f2-6a7a-4b95-9eda-ad0156cc3928 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:25.408129 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca36c46-a717-442e-ab8f-2b3dbc3214ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:25.408816 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca36c46-a717-442e-ab8f-2b3dbc3214ad admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:25.409190 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca36c46-a717-442e-ab8f-2b3dbc3214ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:25.409525 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca36c46-a717-442e-ab8f-2b3dbc3214ad admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:25.410218 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca36c46-a717-442e-ab8f-2b3dbc3214ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:25.412377 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca36c46-a717-442e-ab8f-2b3dbc3214ad admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:25.525290 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 916/1841] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 08:42:25 2022] POST /identity/v3/users => generated 297 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:42:25.528881 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd9ad75-3c95-4fff-a4da-6096fe5a13d3 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:25.540042 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd9ad75-3c95-4fff-a4da-6096fe5a13d3 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:25.546446 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2395841c-6093-44b7-94d6-1da8cdd78a53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:25.548977 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2395841c-6093-44b7-94d6-1da8cdd78a53 admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:25.549335 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2395841c-6093-44b7-94d6-1da8cdd78a53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:25.549663 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2395841c-6093-44b7-94d6-1da8cdd78a53 admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:25.550452 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2395841c-6093-44b7-94d6-1da8cdd78a53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:25.552787 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2395841c-6093-44b7-94d6-1da8cdd78a53 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:25.565131 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 926/1842] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:42:25 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:42:25.573355 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca36c46-a717-442e-ab8f-2b3dbc3214ad admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:25.584444 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca36c46-a717-442e-ab8f-2b3dbc3214ad admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:25.589149 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d46539-1ad9-45b4-aa94-620ca82eac19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:25.589826 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d46539-1ad9-45b4-aa94-620ca82eac19 admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:25.590226 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d46539-1ad9-45b4-aa94-620ca82eac19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:25.590507 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d46539-1ad9-45b4-aa94-620ca82eac19 admin admin] PATH_INFO: `/v3/projects/c58ff7fb2142419a8b9bbeceae23a8b1/users/4ab8a374a42b41ef834e2c4cb318f953/roles/ec3751692d9c4ecf81b4a71c936c46be` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:25.605958 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d46539-1ad9-45b4-aa94-620ca82eac19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c58ff7fb2142419a8b9bbeceae23a8b1, user_id=4ab8a374a42b41ef834e2c4cb318f953, role_id=ec3751692d9c4ecf81b4a71c936c46be)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:25.608800 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d46539-1ad9-45b4-aa94-620ca82eac19 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:25.648851 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:42:25.653538 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 917/1843] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 08:42:25 2022] PUT /identity/v3/projects/c58ff7fb2142419a8b9bbeceae23a8b1/users/4ab8a374a42b41ef834e2c4cb318f953/roles/ec3751692d9c4ecf81b4a71c936c46be => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:42:25.662495 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2395841c-6093-44b7-94d6-1da8cdd78a53 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:25.672905 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2395841c-6093-44b7-94d6-1da8cdd78a53 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:25.726684 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6440bfbe-a008-456e-8f5f-d3ce2fba79e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:25.727210 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6440bfbe-a008-456e-8f5f-d3ce2fba79e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:25.727487 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6440bfbe-a008-456e-8f5f-d3ce2fba79e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:25.727754 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6440bfbe-a008-456e-8f5f-d3ce2fba79e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:25.728376 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6440bfbe-a008-456e-8f5f-d3ce2fba79e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:25.729999 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6440bfbe-a008-456e-8f5f-d3ce2fba79e7 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:25.740213 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 927/1844] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:42:25 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:42:25.744365 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d46539-1ad9-45b4-aa94-620ca82eac19 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:25.755820 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d46539-1ad9-45b4-aa94-620ca82eac19 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:25.760428 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf6db60-082b-47ba-9624-53570bb659d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:25.761139 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf6db60-082b-47ba-9624-53570bb659d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:25.761420 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf6db60-082b-47ba-9624-53570bb659d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:25.761694 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf6db60-082b-47ba-9624-53570bb659d2 admin admin] PATH_INFO: `/v3/projects/c58ff7fb2142419a8b9bbeceae23a8b1/users/4ab8a374a42b41ef834e2c4cb318f953/roles/ec3751692d9c4ecf81b4a71c936c46be` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:25.763621 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf6db60-082b-47ba-9624-53570bb659d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c58ff7fb2142419a8b9bbeceae23a8b1, user_id=4ab8a374a42b41ef834e2c4cb318f953, role_id=ec3751692d9c4ecf81b4a71c936c46be)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:25.766644 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf6db60-082b-47ba-9624-53570bb659d2 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:25.777707 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 918/1845] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 08:42:25 2022] PUT /identity/v3/projects/c58ff7fb2142419a8b9bbeceae23a8b1/users/4ab8a374a42b41ef834e2c4cb318f953/roles/ec3751692d9c4ecf81b4a71c936c46be => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:42:25.788068 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ff94dd-011b-4d36-8834-0ae124adb646 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:25.788353 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ff94dd-011b-4d36-8834-0ae124adb646 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:25.788657 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ff94dd-011b-4d36-8834-0ae124adb646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:25.817775 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-35ff94dd-011b-4d36-8834-0ae124adb646 None None] MFA Rules not processed for user `4ab8a374a42b41ef834e2c4cb318f953`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:42:25.818969 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-35ff94dd-011b-4d36-8834-0ae124adb646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:42:25.885780 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 928/1846] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:42:25 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:42:25.898805 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf6db60-082b-47ba-9624-53570bb659d2 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:25.900566 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-acf6db60-082b-47ba-9624-53570bb659d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:42:25.960290 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf6db60-082b-47ba-9624-53570bb659d2 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:25.964113 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3599d58b-fce7-4c39-9ac8-4d96f1581881 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ba74e28ced4cbba857f54222dfbe41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:25.964701 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3599d58b-fce7-4c39-9ac8-4d96f1581881 service cinder] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:25.965003 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3599d58b-fce7-4c39-9ac8-4d96f1581881 service cinder] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:25.965274 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3599d58b-fce7-4c39-9ac8-4d96f1581881 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:25.975104 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3599d58b-fce7-4c39-9ac8-4d96f1581881 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:25.976910 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3599d58b-fce7-4c39-9ac8-4d96f1581881 service cinder] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:25.986960 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 919/1847] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:42:25 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:42:26.966193 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ff94dd-011b-4d36-8834-0ae124adb646 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:26.976163 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ff94dd-011b-4d36-8834-0ae124adb646 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:26.980844 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd042b0-f995-478e-abed-c5b41e63a122 tempest-VolumeQuotaClassesTest-1850840132 tempest-VolumeQuotaClassesTest-1850840132-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ab8a374a42b41ef834e2c4cb318f953', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c58ff7fb2142419a8b9bbeceae23a8b1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:26.981491 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd042b0-f995-478e-abed-c5b41e63a122 tempest-VolumeQuotaClassesTest-1850840132 tempest-VolumeQuotaClassesTest-1850840132-project-admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:26.981774 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd042b0-f995-478e-abed-c5b41e63a122 tempest-VolumeQuotaClassesTest-1850840132 tempest-VolumeQuotaClassesTest-1850840132-project-admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:26.982049 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd042b0-f995-478e-abed-c5b41e63a122 tempest-VolumeQuotaClassesTest-1850840132 tempest-VolumeQuotaClassesTest-1850840132-project-admin] PATH_INFO: `/v3/domains` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:26.982788 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd042b0-f995-478e-abed-c5b41e63a122 tempest-VolumeQuotaClassesTest-1850840132 tempest-VolumeQuotaClassesTest-1850840132-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:42:26.982996 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd042b0-f995-478e-abed-c5b41e63a122 tempest-VolumeQuotaClassesTest-1850840132 tempest-VolumeQuotaClassesTest-1850840132-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:26.990249 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd042b0-f995-478e-abed-c5b41e63a122 tempest-VolumeQuotaClassesTest-1850840132 tempest-VolumeQuotaClassesTest-1850840132-project-admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:27.000831 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 929/1848] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Aug 10 08:42:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:42:27.005258 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3599d58b-fce7-4c39-9ac8-4d96f1581881 service cinder] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:27.014404 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3599d58b-fce7-4c39-9ac8-4d96f1581881 service cinder] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:27.019732 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cd5f99-1dbd-413d-aef7-b9ca61b71edb tempest-VolumeQuotaClassesTest-1850840132 tempest-VolumeQuotaClassesTest-1850840132-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ab8a374a42b41ef834e2c4cb318f953', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c58ff7fb2142419a8b9bbeceae23a8b1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:27.020349 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cd5f99-1dbd-413d-aef7-b9ca61b71edb tempest-VolumeQuotaClassesTest-1850840132 tempest-VolumeQuotaClassesTest-1850840132-project-admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:27.020632 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cd5f99-1dbd-413d-aef7-b9ca61b71edb tempest-VolumeQuotaClassesTest-1850840132 tempest-VolumeQuotaClassesTest-1850840132-project-admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:27.020906 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cd5f99-1dbd-413d-aef7-b9ca61b71edb tempest-VolumeQuotaClassesTest-1850840132 tempest-VolumeQuotaClassesTest-1850840132-project-admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:27.021570 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cd5f99-1dbd-413d-aef7-b9ca61b71edb tempest-VolumeQuotaClassesTest-1850840132 tempest-VolumeQuotaClassesTest-1850840132-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:27.023465 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cd5f99-1dbd-413d-aef7-b9ca61b71edb tempest-VolumeQuotaClassesTest-1850840132 tempest-VolumeQuotaClassesTest-1850840132-project-admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:27.083048 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 920/1849] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 08:42:27 2022] POST /identity/v3/projects => generated 355 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 08:42:27.087009 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd042b0-f995-478e-abed-c5b41e63a122 tempest-VolumeQuotaClassesTest-1850840132 tempest-VolumeQuotaClassesTest-1850840132-project-admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:27.097751 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd042b0-f995-478e-abed-c5b41e63a122 tempest-VolumeQuotaClassesTest-1850840132 tempest-VolumeQuotaClassesTest-1850840132-project-admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:27.127472 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:42:27.153224 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fe6d92-cb2b-4e0d-8584-c8e0d86068a8 tempest-VolumeQuotaClassesTest-1850840132 tempest-VolumeQuotaClassesTest-1850840132-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ab8a374a42b41ef834e2c4cb318f953', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c58ff7fb2142419a8b9bbeceae23a8b1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:27.153830 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fe6d92-cb2b-4e0d-8584-c8e0d86068a8 tempest-VolumeQuotaClassesTest-1850840132 tempest-VolumeQuotaClassesTest-1850840132-project-admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:27.154113 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fe6d92-cb2b-4e0d-8584-c8e0d86068a8 tempest-VolumeQuotaClassesTest-1850840132 tempest-VolumeQuotaClassesTest-1850840132-project-admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:27.154389 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fe6d92-cb2b-4e0d-8584-c8e0d86068a8 tempest-VolumeQuotaClassesTest-1850840132 tempest-VolumeQuotaClassesTest-1850840132-project-admin] PATH_INFO: `/v3/domains` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:27.155026 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fe6d92-cb2b-4e0d-8584-c8e0d86068a8 tempest-VolumeQuotaClassesTest-1850840132 tempest-VolumeQuotaClassesTest-1850840132-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:42:27.155331 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fe6d92-cb2b-4e0d-8584-c8e0d86068a8 tempest-VolumeQuotaClassesTest-1850840132 tempest-VolumeQuotaClassesTest-1850840132-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:27.157731 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fe6d92-cb2b-4e0d-8584-c8e0d86068a8 tempest-VolumeQuotaClassesTest-1850840132 tempest-VolumeQuotaClassesTest-1850840132-project-admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:27.169329 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 930/1850] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Aug 10 08:42:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:42:27.209159 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cd5f99-1dbd-413d-aef7-b9ca61b71edb tempest-VolumeQuotaClassesTest-1850840132 tempest-VolumeQuotaClassesTest-1850840132-project-admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:27.219990 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cd5f99-1dbd-413d-aef7-b9ca61b71edb tempest-VolumeQuotaClassesTest-1850840132 tempest-VolumeQuotaClassesTest-1850840132-project-admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:27.224503 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36b87a6-09d3-4ce7-8ae9-e50dae9661d2 tempest-VolumeQuotaClassesTest-1850840132 tempest-VolumeQuotaClassesTest-1850840132-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ab8a374a42b41ef834e2c4cb318f953', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c58ff7fb2142419a8b9bbeceae23a8b1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:27.225145 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36b87a6-09d3-4ce7-8ae9-e50dae9661d2 tempest-VolumeQuotaClassesTest-1850840132 tempest-VolumeQuotaClassesTest-1850840132-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:27.225425 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36b87a6-09d3-4ce7-8ae9-e50dae9661d2 tempest-VolumeQuotaClassesTest-1850840132 tempest-VolumeQuotaClassesTest-1850840132-project-admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:27.225691 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36b87a6-09d3-4ce7-8ae9-e50dae9661d2 tempest-VolumeQuotaClassesTest-1850840132 tempest-VolumeQuotaClassesTest-1850840132-project-admin] PATH_INFO: `/v3/projects/5f045afad57d4f40b96914a214641fe3` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:27.226809 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36b87a6-09d3-4ce7-8ae9-e50dae9661d2 tempest-VolumeQuotaClassesTest-1850840132 tempest-VolumeQuotaClassesTest-1850840132-project-admin] RBAC: Authorizing `identity:delete_project(project_id=5f045afad57d4f40b96914a214641fe3)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:27.228728 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36b87a6-09d3-4ce7-8ae9-e50dae9661d2 tempest-VolumeQuotaClassesTest-1850840132 tempest-VolumeQuotaClassesTest-1850840132-project-admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:27.327095 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-d36b87a6-09d3-4ce7-8ae9-e50dae9661d2 tempest-VolumeQuotaClassesTest-1850840132 tempest-VolumeQuotaClassesTest-1850840132-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f045afad57d4f40b96914a214641fe3', 'request_id': 'req-d36b87a6-09d3-4ce7-8ae9-e50dae9661d2'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:27.398356 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-d36b87a6-09d3-4ce7-8ae9-e50dae9661d2 tempest-VolumeQuotaClassesTest-1850840132 tempest-VolumeQuotaClassesTest-1850840132-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f045afad57d4f40b96914a214641fe3', 'request_id': 'req-d36b87a6-09d3-4ce7-8ae9-e50dae9661d2'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:27.407768 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-d36b87a6-09d3-4ce7-8ae9-e50dae9661d2 tempest-VolumeQuotaClassesTest-1850840132 tempest-VolumeQuotaClassesTest-1850840132-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f045afad57d4f40b96914a214641fe3', 'request_id': 'req-d36b87a6-09d3-4ce7-8ae9-e50dae9661d2'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:27.420349 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-d36b87a6-09d3-4ce7-8ae9-e50dae9661d2 tempest-VolumeQuotaClassesTest-1850840132 tempest-VolumeQuotaClassesTest-1850840132-project-admin] The token cache is being invalidate because project 5f045afad57d4f40b96914a214641fe3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 08:42:27.420640 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-d36b87a6-09d3-4ce7-8ae9-e50dae9661d2 tempest-VolumeQuotaClassesTest-1850840132 tempest-VolumeQuotaClassesTest-1850840132-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:27.421405 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-d36b87a6-09d3-4ce7-8ae9-e50dae9661d2 tempest-VolumeQuotaClassesTest-1850840132 tempest-VolumeQuotaClassesTest-1850840132-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:27.422898 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 921/1851] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 08:42:27 2022] DELETE /identity/v3/projects/5f045afad57d4f40b96914a214641fe3 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:42:28.079628 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fe6d92-cb2b-4e0d-8584-c8e0d86068a8 tempest-VolumeQuotaClassesTest-1850840132 tempest-VolumeQuotaClassesTest-1850840132-project-admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:28.081561 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-26fe6d92-cb2b-4e0d-8584-c8e0d86068a8 tempest-VolumeQuotaClassesTest-1850840132 tempest-VolumeQuotaClassesTest-1850840132-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:42:28.149676 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fe6d92-cb2b-4e0d-8584-c8e0d86068a8 tempest-VolumeQuotaClassesTest-1850840132 tempest-VolumeQuotaClassesTest-1850840132-project-admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:28.204530 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a6211c-e68d-43f7-a316-0df4cbc9f641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:28.205176 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a6211c-e68d-43f7-a316-0df4cbc9f641 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:28.205460 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a6211c-e68d-43f7-a316-0df4cbc9f641 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:28.205732 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a6211c-e68d-43f7-a316-0df4cbc9f641 admin admin] PATH_INFO: `/v3/users/1cad005805034957bbfabbad8e0f547e` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:28.206872 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a6211c-e68d-43f7-a316-0df4cbc9f641 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1cad005805034957bbfabbad8e0f547e)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:28.209581 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a6211c-e68d-43f7-a316-0df4cbc9f641 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:28.374927 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-08a6211c-e68d-43f7-a316-0df4cbc9f641 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1cad005805034957bbfabbad8e0f547e'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:28.387875 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-08a6211c-e68d-43f7-a316-0df4cbc9f641 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1cad005805034957bbfabbad8e0f547e'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:28.395609 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-08a6211c-e68d-43f7-a316-0df4cbc9f641 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1cad005805034957bbfabbad8e0f547e'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:28.398004 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:42:28.492834 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-08a6211c-e68d-43f7-a316-0df4cbc9f641 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1cad005805034957bbfabbad8e0f547e'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:28.493576 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-08a6211c-e68d-43f7-a316-0df4cbc9f641 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1cad005805034957bbfabbad8e0f547e'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:28.500159 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 931/1852] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 08:42:28 2022] DELETE /identity/v3/users/1cad005805034957bbfabbad8e0f547e => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:42:28.505328 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362ac14d-7f01-4b48-a71f-9a52b586ce5c None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:28.505807 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362ac14d-7f01-4b48-a71f-9a52b586ce5c None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:28.505966 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362ac14d-7f01-4b48-a71f-9a52b586ce5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:28.525724 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-362ac14d-7f01-4b48-a71f-9a52b586ce5c None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:42:28.527716 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-362ac14d-7f01-4b48-a71f-9a52b586ce5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:42:28.573866 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:42:28.587940 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 922/1853] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:42:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 08:42:28.597765 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a6211c-e68d-43f7-a316-0df4cbc9f641 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:28.599219 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-08a6211c-e68d-43f7-a316-0df4cbc9f641 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:42:28.656398 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a6211c-e68d-43f7-a316-0df4cbc9f641 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:28.660080 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926e72de-48c0-4d7b-9487-50232ffcea27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:28.660749 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926e72de-48c0-4d7b-9487-50232ffcea27 service neutron] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:28.661058 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926e72de-48c0-4d7b-9487-50232ffcea27 service neutron] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:28.661329 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926e72de-48c0-4d7b-9487-50232ffcea27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:28.670118 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926e72de-48c0-4d7b-9487-50232ffcea27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:28.671543 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926e72de-48c0-4d7b-9487-50232ffcea27 service neutron] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:28.681752 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 932/1854] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:42:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:42:29.131249 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362ac14d-7f01-4b48-a71f-9a52b586ce5c None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:29.133069 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-362ac14d-7f01-4b48-a71f-9a52b586ce5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:42:29.149316 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362ac14d-7f01-4b48-a71f-9a52b586ce5c None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:29.155024 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a72d3f-863d-4a43-9372-7f38bd69b399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:29.155832 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a72d3f-863d-4a43-9372-7f38bd69b399 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:29.156512 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a72d3f-863d-4a43-9372-7f38bd69b399 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:29.156885 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a72d3f-863d-4a43-9372-7f38bd69b399 admin admin] PATH_INFO: `/v3/projects/525e60831c22404d99d2ca9a378fd3a9` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:29.158265 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a72d3f-863d-4a43-9372-7f38bd69b399 admin admin] RBAC: Authorizing `identity:delete_project(project_id=525e60831c22404d99d2ca9a378fd3a9)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:29.161338 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a72d3f-863d-4a43-9372-7f38bd69b399 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:29.272047 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-32a72d3f-863d-4a43-9372-7f38bd69b399 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '525e60831c22404d99d2ca9a378fd3a9', 'request_id': 'req-32a72d3f-863d-4a43-9372-7f38bd69b399'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:29.333248 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-32a72d3f-863d-4a43-9372-7f38bd69b399 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '525e60831c22404d99d2ca9a378fd3a9', 'request_id': 'req-32a72d3f-863d-4a43-9372-7f38bd69b399'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:29.339524 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-32a72d3f-863d-4a43-9372-7f38bd69b399 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '525e60831c22404d99d2ca9a378fd3a9', 'request_id': 'req-32a72d3f-863d-4a43-9372-7f38bd69b399'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:29.351508 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-32a72d3f-863d-4a43-9372-7f38bd69b399 admin admin] The token cache is being invalidate because project 525e60831c22404d99d2ca9a378fd3a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 08:42:29.351843 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-32a72d3f-863d-4a43-9372-7f38bd69b399 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:29.352498 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-32a72d3f-863d-4a43-9372-7f38bd69b399 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:29.357010 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 923/1855] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 08:42:29 2022] DELETE /identity/v3/projects/525e60831c22404d99d2ca9a378fd3a9 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:42:29.360891 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926e72de-48c0-4d7b-9487-50232ffcea27 service neutron] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:29.371136 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926e72de-48c0-4d7b-9487-50232ffcea27 service neutron] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:29.429494 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6b131a-9802-448b-b244-89eff6884dba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:29.430133 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6b131a-9802-448b-b244-89eff6884dba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:29.430414 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6b131a-9802-448b-b244-89eff6884dba admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:29.430690 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6b131a-9802-448b-b244-89eff6884dba admin admin] PATH_INFO: `/v3/users/4ab8a374a42b41ef834e2c4cb318f953` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:29.431866 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6b131a-9802-448b-b244-89eff6884dba admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ab8a374a42b41ef834e2c4cb318f953)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:29.433877 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6b131a-9802-448b-b244-89eff6884dba admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:29.655669 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-4f6b131a-9802-448b-b244-89eff6884dba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ab8a374a42b41ef834e2c4cb318f953'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:29.667824 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-4f6b131a-9802-448b-b244-89eff6884dba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ab8a374a42b41ef834e2c4cb318f953'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:29.674498 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-4f6b131a-9802-448b-b244-89eff6884dba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ab8a374a42b41ef834e2c4cb318f953'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:29.727161 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-4f6b131a-9802-448b-b244-89eff6884dba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ab8a374a42b41ef834e2c4cb318f953'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:29.727803 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-4f6b131a-9802-448b-b244-89eff6884dba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ab8a374a42b41ef834e2c4cb318f953'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:29.733812 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 933/1856] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 08:42:29 2022] DELETE /identity/v3/users/4ab8a374a42b41ef834e2c4cb318f953 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:42:30.211013 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a72d3f-863d-4a43-9372-7f38bd69b399 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:30.213905 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-32a72d3f-863d-4a43-9372-7f38bd69b399 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:42:30.236483 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a72d3f-863d-4a43-9372-7f38bd69b399 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:30.304870 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b612dc9-3053-4c33-9906-731d2a96d280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:30.305543 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b612dc9-3053-4c33-9906-731d2a96d280 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:30.305827 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b612dc9-3053-4c33-9906-731d2a96d280 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:30.306104 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b612dc9-3053-4c33-9906-731d2a96d280 admin admin] PATH_INFO: `/v3/projects/c58ff7fb2142419a8b9bbeceae23a8b1` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:30.307247 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b612dc9-3053-4c33-9906-731d2a96d280 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c58ff7fb2142419a8b9bbeceae23a8b1)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:30.309311 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b612dc9-3053-4c33-9906-731d2a96d280 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:30.420446 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-8b612dc9-3053-4c33-9906-731d2a96d280 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c58ff7fb2142419a8b9bbeceae23a8b1', 'request_id': 'req-8b612dc9-3053-4c33-9906-731d2a96d280'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:30.510447 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-8b612dc9-3053-4c33-9906-731d2a96d280 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c58ff7fb2142419a8b9bbeceae23a8b1', 'request_id': 'req-8b612dc9-3053-4c33-9906-731d2a96d280'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:30.516950 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-8b612dc9-3053-4c33-9906-731d2a96d280 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c58ff7fb2142419a8b9bbeceae23a8b1', 'request_id': 'req-8b612dc9-3053-4c33-9906-731d2a96d280'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:30.530550 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-8b612dc9-3053-4c33-9906-731d2a96d280 admin admin] The token cache is being invalidate because project c58ff7fb2142419a8b9bbeceae23a8b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 08:42:30.530924 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-8b612dc9-3053-4c33-9906-731d2a96d280 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:30.531769 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-8b612dc9-3053-4c33-9906-731d2a96d280 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:30.533364 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 924/1857] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 08:42:30 2022] DELETE /identity/v3/projects/c58ff7fb2142419a8b9bbeceae23a8b1 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:42:30.539668 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bbdfeb-9d2b-4db6-86f2-6f4edb06d6c3 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:30.542724 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bbdfeb-9d2b-4db6-86f2-6f4edb06d6c3 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:30.542724 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bbdfeb-9d2b-4db6-86f2-6f4edb06d6c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:30.560095 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-a1bbdfeb-9d2b-4db6-86f2-6f4edb06d6c3 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:42:30.561333 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-a1bbdfeb-9d2b-4db6-86f2-6f4edb06d6c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:42:30.572063 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:42:30.623240 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 934/1858] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:42:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 08:42:30.632745 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6056751-bbe0-424f-8db1-4941dee707b6 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:30.633068 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6056751-bbe0-424f-8db1-4941dee707b6 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:30.633344 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6056751-bbe0-424f-8db1-4941dee707b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:30.642163 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:42:30.653976 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-e6056751-bbe0-424f-8db1-4941dee707b6 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:42:30.655225 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-e6056751-bbe0-424f-8db1-4941dee707b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:42:30.663236 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 925/1859] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:42:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:42:30.667646 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bbdfeb-9d2b-4db6-86f2-6f4edb06d6c3 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:30.678809 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bbdfeb-9d2b-4db6-86f2-6f4edb06d6c3 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:30.683588 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc970133-4253-419c-9265-93d5c33a6380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:30.684222 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc970133-4253-419c-9265-93d5c33a6380 admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:30.684507 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc970133-4253-419c-9265-93d5c33a6380 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:30.684782 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc970133-4253-419c-9265-93d5c33a6380 admin admin] PATH_INFO: `/v3/domains` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:30.685515 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc970133-4253-419c-9265-93d5c33a6380 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:42:30.685822 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc970133-4253-419c-9265-93d5c33a6380 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:30.687539 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc970133-4253-419c-9265-93d5c33a6380 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:30.700601 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 935/1860] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Aug 10 08:42:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:42:30.709186 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6056751-bbe0-424f-8db1-4941dee707b6 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:30.718551 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6056751-bbe0-424f-8db1-4941dee707b6 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:30.723795 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d81b9bb-2160-43f8-a694-cd78fb8584f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:30.724416 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d81b9bb-2160-43f8-a694-cd78fb8584f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:30.724724 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d81b9bb-2160-43f8-a694-cd78fb8584f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:30.725080 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d81b9bb-2160-43f8-a694-cd78fb8584f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:30.725653 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d81b9bb-2160-43f8-a694-cd78fb8584f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:30.727550 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d81b9bb-2160-43f8-a694-cd78fb8584f3 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:30.835123 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 926/1861] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 08:42:30 2022] POST /identity/v3/projects => generated 375 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 08:42:30.836516 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc970133-4253-419c-9265-93d5c33a6380 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:30.846729 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc970133-4253-419c-9265-93d5c33a6380 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:30.921413 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101d0db0-b390-4986-937c-fe0684cec1ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:30.922030 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101d0db0-b390-4986-937c-fe0684cec1ac admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:30.922311 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101d0db0-b390-4986-937c-fe0684cec1ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:30.922580 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101d0db0-b390-4986-937c-fe0684cec1ac admin admin] PATH_INFO: `/v3/users` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:30.923203 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101d0db0-b390-4986-937c-fe0684cec1ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:30.925203 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101d0db0-b390-4986-937c-fe0684cec1ac admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:31.036454 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 936/1862] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 08:42:30 2022] POST /identity/v3/users => generated 288 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:42:31.040398 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d81b9bb-2160-43f8-a694-cd78fb8584f3 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:31.050369 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d81b9bb-2160-43f8-a694-cd78fb8584f3 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:31.055921 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f41cb80-6fef-432f-969b-1a1931d3eeb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:31.056511 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f41cb80-6fef-432f-969b-1a1931d3eeb9 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:31.056783 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f41cb80-6fef-432f-969b-1a1931d3eeb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:31.057082 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f41cb80-6fef-432f-969b-1a1931d3eeb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:31.057741 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f41cb80-6fef-432f-969b-1a1931d3eeb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:31.059336 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f41cb80-6fef-432f-969b-1a1931d3eeb9 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:31.070536 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 927/1863] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:42:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:42:31.074593 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101d0db0-b390-4986-937c-fe0684cec1ac admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:31.085144 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101d0db0-b390-4986-937c-fe0684cec1ac admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:31.090090 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83891c8-aa9a-4d97-a5e3-0f4eed638064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:31.090699 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83891c8-aa9a-4d97-a5e3-0f4eed638064 admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:31.090993 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83891c8-aa9a-4d97-a5e3-0f4eed638064 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:31.091253 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83891c8-aa9a-4d97-a5e3-0f4eed638064 admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:31.091895 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83891c8-aa9a-4d97-a5e3-0f4eed638064 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:31.093567 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83891c8-aa9a-4d97-a5e3-0f4eed638064 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:31.107162 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 937/1864] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:42:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:42:31.111237 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f41cb80-6fef-432f-969b-1a1931d3eeb9 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:31.123152 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f41cb80-6fef-432f-969b-1a1931d3eeb9 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:31.127964 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18002c3d-c9ec-4732-89e7-eaead60f5837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:31.128618 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18002c3d-c9ec-4732-89e7-eaead60f5837 admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:31.128984 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18002c3d-c9ec-4732-89e7-eaead60f5837 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:31.129326 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18002c3d-c9ec-4732-89e7-eaead60f5837 admin admin] PATH_INFO: `/v3/projects/c547b4ce9248430a9fbccf818c7203b3/users/afc19691612448889ba6f41d666a98a0/roles/44b64db484f24a46bf0b7ced46b519bf` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:31.144348 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18002c3d-c9ec-4732-89e7-eaead60f5837 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c547b4ce9248430a9fbccf818c7203b3, user_id=afc19691612448889ba6f41d666a98a0, role_id=44b64db484f24a46bf0b7ced46b519bf)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:31.147388 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18002c3d-c9ec-4732-89e7-eaead60f5837 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:31.232628 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 928/1865] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 08:42:31 2022] PUT /identity/v3/projects/c547b4ce9248430a9fbccf818c7203b3/users/afc19691612448889ba6f41d666a98a0/roles/44b64db484f24a46bf0b7ced46b519bf => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:42:31.246194 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d12d75d-ab8d-4c94-b93b-0bd7f8090e7e None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:31.249892 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d12d75d-ab8d-4c94-b93b-0bd7f8090e7e None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:31.249892 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d12d75d-ab8d-4c94-b93b-0bd7f8090e7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:31.277639 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-3d12d75d-ab8d-4c94-b93b-0bd7f8090e7e None None] MFA Rules not processed for user `afc19691612448889ba6f41d666a98a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:42:31.278836 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-3d12d75d-ab8d-4c94-b93b-0bd7f8090e7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:42:31.361065 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 938/1866] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:42:31 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:42:31.366275 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18002c3d-c9ec-4732-89e7-eaead60f5837 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:31.440648 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18002c3d-c9ec-4732-89e7-eaead60f5837 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:31.446210 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14d08e5-2cc3-48fc-b4fa-d2ccb2bafa34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:31.446822 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14d08e5-2cc3-48fc-b4fa-d2ccb2bafa34 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:31.447098 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14d08e5-2cc3-48fc-b4fa-d2ccb2bafa34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:31.447366 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14d08e5-2cc3-48fc-b4fa-d2ccb2bafa34 admin admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:31.448011 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14d08e5-2cc3-48fc-b4fa-d2ccb2bafa34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:31.450615 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14d08e5-2cc3-48fc-b4fa-d2ccb2bafa34 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:31.561185 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 929/1867] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 08:42:31 2022] POST /identity/v3/projects => generated 377 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:42:31.562930 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d12d75d-ab8d-4c94-b93b-0bd7f8090e7e None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:31.574709 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d12d75d-ab8d-4c94-b93b-0bd7f8090e7e None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:31.583089 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:42:31.652701 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72656d7e-de4e-4484-a862-4df466540801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:31.653407 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72656d7e-de4e-4484-a862-4df466540801 admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:31.653802 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72656d7e-de4e-4484-a862-4df466540801 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:31.654139 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72656d7e-de4e-4484-a862-4df466540801 admin admin] PATH_INFO: `/v3/users` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:31.654825 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72656d7e-de4e-4484-a862-4df466540801 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:31.656806 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72656d7e-de4e-4484-a862-4df466540801 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:31.741658 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 939/1868] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 08:42:31 2022] POST /identity/v3/users => generated 289 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:42:31.745626 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14d08e5-2cc3-48fc-b4fa-d2ccb2bafa34 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:31.761463 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14d08e5-2cc3-48fc-b4fa-d2ccb2bafa34 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:31.774895 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e8a924-dc7c-4764-b595-e7d12d9f3692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:31.774895 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e8a924-dc7c-4764-b595-e7d12d9f3692 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:31.774895 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e8a924-dc7c-4764-b595-e7d12d9f3692 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:31.774895 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e8a924-dc7c-4764-b595-e7d12d9f3692 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:31.775835 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e8a924-dc7c-4764-b595-e7d12d9f3692 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:31.778152 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e8a924-dc7c-4764-b595-e7d12d9f3692 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:31.797125 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 930/1869] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:42:31 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:42:31.800744 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72656d7e-de4e-4484-a862-4df466540801 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:31.831934 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72656d7e-de4e-4484-a862-4df466540801 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:31.845133 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa918d1c-a39d-473a-9325-425ff5250310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:31.846073 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa918d1c-a39d-473a-9325-425ff5250310 admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:31.846539 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa918d1c-a39d-473a-9325-425ff5250310 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:31.846995 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa918d1c-a39d-473a-9325-425ff5250310 admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:31.848704 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa918d1c-a39d-473a-9325-425ff5250310 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:31.850954 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa918d1c-a39d-473a-9325-425ff5250310 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:31.864286 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 940/1870] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:42:31 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:42:31.868804 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e8a924-dc7c-4764-b595-e7d12d9f3692 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:31.870892 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38056c7-2795-4a0b-a67e-278ae8f25c74 None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:31.871507 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38056c7-2795-4a0b-a67e-278ae8f25c74 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:31.871925 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38056c7-2795-4a0b-a67e-278ae8f25c74 None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:31.877440 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 941/1871] 10.222.0.57 () {54 vars in 921 bytes} [Wed Aug 10 08:42:31 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:42:31.879453 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e8a924-dc7c-4764-b595-e7d12d9f3692 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:31.884834 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd210770-ee66-4af8-84d2-8cc1bbfc9caa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:31.885530 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd210770-ee66-4af8-84d2-8cc1bbfc9caa admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:31.885808 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd210770-ee66-4af8-84d2-8cc1bbfc9caa admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:31.886078 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd210770-ee66-4af8-84d2-8cc1bbfc9caa admin admin] PATH_INFO: `/v3/projects/c9ea639f8d1543b9b884cff530230ea8/users/2e6de0376fb34cd981b3a4c485f25a8d/roles/44b64db484f24a46bf0b7ced46b519bf` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:31.889696 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9172d86-1448-4ef2-9a86-c0d2603390b6 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:31.890149 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9172d86-1448-4ef2-9a86-c0d2603390b6 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:31.890427 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9172d86-1448-4ef2-9a86-c0d2603390b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:31.915780 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-c9172d86-1448-4ef2-9a86-c0d2603390b6 None None] MFA Rules not processed for user `a943008193914f76a663d489bd73e873`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:42:31.917254 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-c9172d86-1448-4ef2-9a86-c0d2603390b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:42:31.929800 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd210770-ee66-4af8-84d2-8cc1bbfc9caa admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9ea639f8d1543b9b884cff530230ea8, user_id=2e6de0376fb34cd981b3a4c485f25a8d, role_id=44b64db484f24a46bf0b7ced46b519bf)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:31.932473 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd210770-ee66-4af8-84d2-8cc1bbfc9caa admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:32.026987 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 931/1872] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 08:42:31 2022] PUT /identity/v3/projects/c9ea639f8d1543b9b884cff530230ea8/users/2e6de0376fb34cd981b3a4c485f25a8d/roles/44b64db484f24a46bf0b7ced46b519bf => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:42:32.028874 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6270fe7-8ccf-42dd-91c0-eedf67a05c91 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:32.029336 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6270fe7-8ccf-42dd-91c0-eedf67a05c91 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:32.029750 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6270fe7-8ccf-42dd-91c0-eedf67a05c91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:32.030252 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 942/1873] 10.222.0.57 () {58 vars in 1018 bytes} [Wed Aug 10 08:42:31 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:42:32.081303 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-c6270fe7-8ccf-42dd-91c0-eedf67a05c91 None None] MFA Rules not processed for user `2e6de0376fb34cd981b3a4c485f25a8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:42:32.083361 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-c6270fe7-8ccf-42dd-91c0-eedf67a05c91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:42:32.118278 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9172d86-1448-4ef2-9a86-c0d2603390b6 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:32.119876 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-c9172d86-1448-4ef2-9a86-c0d2603390b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:42:32.141269 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9172d86-1448-4ef2-9a86-c0d2603390b6 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:32.163375 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:42:32.194248 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 932/1874] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:42:32 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:42:32.198741 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6270fe7-8ccf-42dd-91c0-eedf67a05c91 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:32.219171 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1ae112-8587-4c60-9cfd-5a043d606b82 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a943008193914f76a663d489bd73e873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:32.219806 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1ae112-8587-4c60-9cfd-5a043d606b82 service nova] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:32.220197 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1ae112-8587-4c60-9cfd-5a043d606b82 service nova] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:32.220465 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1ae112-8587-4c60-9cfd-5a043d606b82 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:32.229353 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1ae112-8587-4c60-9cfd-5a043d606b82 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:32.230961 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1ae112-8587-4c60-9cfd-5a043d606b82 service nova] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:32.242404 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 943/1875] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:42:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 08:42:32.261982 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6270fe7-8ccf-42dd-91c0-eedf67a05c91 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:32.266549 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba2274c-780c-47be-a5ee-4483b9350a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:32.267154 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba2274c-780c-47be-a5ee-4483b9350a27 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:32.267435 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba2274c-780c-47be-a5ee-4483b9350a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:32.267705 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba2274c-780c-47be-a5ee-4483b9350a27 admin admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:32.268360 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba2274c-780c-47be-a5ee-4483b9350a27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:32.270410 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba2274c-780c-47be-a5ee-4483b9350a27 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:32.384740 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 933/1876] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 08:42:32 2022] POST /identity/v3/projects => generated 377 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 08:42:32.387455 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1ae112-8587-4c60-9cfd-5a043d606b82 service nova] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:32.398626 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1ae112-8587-4c60-9cfd-5a043d606b82 service nova] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:32.402858 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:42:32.456740 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8086892f-2d65-4d7e-9822-d757a72bdaf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:32.457858 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8086892f-2d65-4d7e-9822-d757a72bdaf2 admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:32.458344 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8086892f-2d65-4d7e-9822-d757a72bdaf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:32.458788 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8086892f-2d65-4d7e-9822-d757a72bdaf2 admin admin] PATH_INFO: `/v3/users` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:32.459769 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8086892f-2d65-4d7e-9822-d757a72bdaf2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:32.461861 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8086892f-2d65-4d7e-9822-d757a72bdaf2 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:32.587664 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 944/1877] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 08:42:32 2022] POST /identity/v3/users => generated 295 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:42:32.588998 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba2274c-780c-47be-a5ee-4483b9350a27 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:32.599640 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba2274c-780c-47be-a5ee-4483b9350a27 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:32.604113 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8b1bed-25a3-4b02-88d4-d9302dfee5c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:32.604710 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8b1bed-25a3-4b02-88d4-d9302dfee5c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:32.605024 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8b1bed-25a3-4b02-88d4-d9302dfee5c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:32.605298 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8b1bed-25a3-4b02-88d4-d9302dfee5c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:32.606007 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8b1bed-25a3-4b02-88d4-d9302dfee5c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:32.607584 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8b1bed-25a3-4b02-88d4-d9302dfee5c7 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:32.618135 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 934/1878] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:42:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 08:42:32.622168 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8086892f-2d65-4d7e-9822-d757a72bdaf2 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:32.632823 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8086892f-2d65-4d7e-9822-d757a72bdaf2 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:32.637470 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa52ded-c465-4a2d-ac33-2cbf36a58f97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:32.638312 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa52ded-c465-4a2d-ac33-2cbf36a58f97 admin admin] REQUEST_METHOD: `PUT` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:32.638596 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa52ded-c465-4a2d-ac33-2cbf36a58f97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:32.638869 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa52ded-c465-4a2d-ac33-2cbf36a58f97 admin admin] PATH_INFO: `/v3/projects/ef5f4330a7a348c99c6576f4762acc3c/users/865db966e32e44cba734f7ff743006d4/roles/ec3751692d9c4ecf81b4a71c936c46be` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:32.656212 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa52ded-c465-4a2d-ac33-2cbf36a58f97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef5f4330a7a348c99c6576f4762acc3c, user_id=865db966e32e44cba734f7ff743006d4, role_id=ec3751692d9c4ecf81b4a71c936c46be)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:32.659432 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa52ded-c465-4a2d-ac33-2cbf36a58f97 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:32.710893 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 945/1879] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 08:42:32 2022] PUT /identity/v3/projects/ef5f4330a7a348c99c6576f4762acc3c/users/865db966e32e44cba734f7ff743006d4/roles/ec3751692d9c4ecf81b4a71c936c46be => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:42:32.714936 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8b1bed-25a3-4b02-88d4-d9302dfee5c7 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:32.729038 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8b1bed-25a3-4b02-88d4-d9302dfee5c7 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:32.790050 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3add28-6d02-42fe-834c-770d17ac1d22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:32.790643 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3add28-6d02-42fe-834c-770d17ac1d22 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:32.790923 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3add28-6d02-42fe-834c-770d17ac1d22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:32.791194 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3add28-6d02-42fe-834c-770d17ac1d22 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:32.791837 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3add28-6d02-42fe-834c-770d17ac1d22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:32.793469 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3add28-6d02-42fe-834c-770d17ac1d22 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:32.807541 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 935/1880] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:42:32 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:42:32.808767 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa52ded-c465-4a2d-ac33-2cbf36a58f97 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:32.819077 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa52ded-c465-4a2d-ac33-2cbf36a58f97 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:32.824824 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fe9fd3-5944-4667-a967-2da388fe4957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:32.825567 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fe9fd3-5944-4667-a967-2da388fe4957 admin admin] REQUEST_METHOD: `PUT` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:32.825848 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fe9fd3-5944-4667-a967-2da388fe4957 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:32.826120 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fe9fd3-5944-4667-a967-2da388fe4957 admin admin] PATH_INFO: `/v3/projects/ef5f4330a7a348c99c6576f4762acc3c/users/865db966e32e44cba734f7ff743006d4/roles/ec3751692d9c4ecf81b4a71c936c46be` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:32.828109 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fe9fd3-5944-4667-a967-2da388fe4957 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef5f4330a7a348c99c6576f4762acc3c, user_id=865db966e32e44cba734f7ff743006d4, role_id=ec3751692d9c4ecf81b4a71c936c46be)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:32.830839 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fe9fd3-5944-4667-a967-2da388fe4957 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:32.841551 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 946/1881] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 08:42:32 2022] PUT /identity/v3/projects/ef5f4330a7a348c99c6576f4762acc3c/users/865db966e32e44cba734f7ff743006d4/roles/ec3751692d9c4ecf81b4a71c936c46be => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:42:32.858616 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336cbab6-1ba8-4097-8673-6ef89c1feeb1 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:32.858616 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336cbab6-1ba8-4097-8673-6ef89c1feeb1 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:32.858616 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336cbab6-1ba8-4097-8673-6ef89c1feeb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:32.887428 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-336cbab6-1ba8-4097-8673-6ef89c1feeb1 None None] MFA Rules not processed for user `865db966e32e44cba734f7ff743006d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:42:32.888640 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-336cbab6-1ba8-4097-8673-6ef89c1feeb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:42:32.929114 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:42:32.958940 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 936/1882] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:42:32 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 08:42:32.970800 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fe9fd3-5944-4667-a967-2da388fe4957 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:32.972621 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-38fe9fd3-5944-4667-a967-2da388fe4957 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:42:33.033892 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fe9fd3-5944-4667-a967-2da388fe4957 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:33.038300 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d70cf4-1d96-4499-b19b-8453484b1ed6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ba74e28ced4cbba857f54222dfbe41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:33.038903 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d70cf4-1d96-4499-b19b-8453484b1ed6 service cinder] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:33.039178 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d70cf4-1d96-4499-b19b-8453484b1ed6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:33.039443 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d70cf4-1d96-4499-b19b-8453484b1ed6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:33.097854 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d70cf4-1d96-4499-b19b-8453484b1ed6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:33.099308 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d70cf4-1d96-4499-b19b-8453484b1ed6 service cinder] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:33.109436 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 947/1883] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:42:32 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:42:38.389250 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336cbab6-1ba8-4097-8673-6ef89c1feeb1 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:38.400740 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336cbab6-1ba8-4097-8673-6ef89c1feeb1 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:38.425968 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eabd21-4aff-41b7-a537-31e4afa8417d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ba74e28ced4cbba857f54222dfbe41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:38.425968 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eabd21-4aff-41b7-a537-31e4afa8417d service cinder] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:38.425968 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eabd21-4aff-41b7-a537-31e4afa8417d service cinder] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:38.425968 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eabd21-4aff-41b7-a537-31e4afa8417d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:38.467203 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52eabd21-4aff-41b7-a537-31e4afa8417d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:38.468635 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52eabd21-4aff-41b7-a537-31e4afa8417d service cinder] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:38.482995 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 937/1884] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:42:38 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:42:42.937401 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d70cf4-1d96-4499-b19b-8453484b1ed6 service cinder] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:42.946595 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d70cf4-1d96-4499-b19b-8453484b1ed6 service cinder] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:42.950309 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57946481-7659-45e8-89a8-e7bcd29a0990 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ba74e28ced4cbba857f54222dfbe41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:42.950847 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57946481-7659-45e8-89a8-e7bcd29a0990 service cinder] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:42.951129 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57946481-7659-45e8-89a8-e7bcd29a0990 service cinder] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:42.951398 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57946481-7659-45e8-89a8-e7bcd29a0990 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:42.960366 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57946481-7659-45e8-89a8-e7bcd29a0990 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:42.961817 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57946481-7659-45e8-89a8-e7bcd29a0990 service cinder] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:42.966681 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:42:42.971199 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 948/1885] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:42:42 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:42:46.561946 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eabd21-4aff-41b7-a537-31e4afa8417d service cinder] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:46.563438 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-52eabd21-4aff-41b7-a537-31e4afa8417d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:42:46.629360 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eabd21-4aff-41b7-a537-31e4afa8417d service cinder] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:46.634356 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd980ae-8e26-4549-b23f-af8cb0477efd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:46.634983 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd980ae-8e26-4549-b23f-af8cb0477efd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:46.635264 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd980ae-8e26-4549-b23f-af8cb0477efd admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:46.635535 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd980ae-8e26-4549-b23f-af8cb0477efd admin admin] PATH_INFO: `/v3/users/0bada08693a549749f98680868f6ada1` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:46.636639 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd980ae-8e26-4549-b23f-af8cb0477efd admin admin] RBAC: Authorizing `identity:delete_user(user_id=0bada08693a549749f98680868f6ada1)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:46.638663 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd980ae-8e26-4549-b23f-af8cb0477efd admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:46.793161 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-1dd980ae-8e26-4549-b23f-af8cb0477efd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0bada08693a549749f98680868f6ada1'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:46.806894 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-1dd980ae-8e26-4549-b23f-af8cb0477efd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0bada08693a549749f98680868f6ada1'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:46.825605 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-1dd980ae-8e26-4549-b23f-af8cb0477efd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0bada08693a549749f98680868f6ada1'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:46.826519 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-1dd980ae-8e26-4549-b23f-af8cb0477efd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0bada08693a549749f98680868f6ada1'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:46.827133 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-1dd980ae-8e26-4549-b23f-af8cb0477efd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0bada08693a549749f98680868f6ada1'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:46.835694 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:42:46.924630 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 938/1886] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 08:42:46 2022] DELETE /identity/v3/users/0bada08693a549749f98680868f6ada1 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 08:42:46.928217 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca31f4f6-ad43-4bd3-b8f7-ff1253ab81f7 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:46.928765 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca31f4f6-ad43-4bd3-b8f7-ff1253ab81f7 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:46.928933 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca31f4f6-ad43-4bd3-b8f7-ff1253ab81f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:46.951649 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-ca31f4f6-ad43-4bd3-b8f7-ff1253ab81f7 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:42:46.953423 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-ca31f4f6-ad43-4bd3-b8f7-ff1253ab81f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:42:47.081468 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd980ae-8e26-4549-b23f-af8cb0477efd admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:47.081468 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-1dd980ae-8e26-4549-b23f-af8cb0477efd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:42:47.084311 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 949/1887] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:42:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:42:47.159454 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd980ae-8e26-4549-b23f-af8cb0477efd admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:47.163736 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb1712a-ade0-476f-b697-46017f7f1026 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:47.164345 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb1712a-ade0-476f-b697-46017f7f1026 service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:47.164628 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb1712a-ade0-476f-b697-46017f7f1026 service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:47.167805 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb1712a-ade0-476f-b697-46017f7f1026 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:47.174689 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb1712a-ade0-476f-b697-46017f7f1026 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:47.176136 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb1712a-ade0-476f-b697-46017f7f1026 service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:47.186805 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 939/1888] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:42:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 08:42:47.662907 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca31f4f6-ad43-4bd3-b8f7-ff1253ab81f7 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:47.664810 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-ca31f4f6-ad43-4bd3-b8f7-ff1253ab81f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:42:47.676516 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca31f4f6-ad43-4bd3-b8f7-ff1253ab81f7 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:47.681836 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e323902-93b4-4f7d-8c05-648130742ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:47.682516 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e323902-93b4-4f7d-8c05-648130742ac1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:47.682788 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e323902-93b4-4f7d-8c05-648130742ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:47.683055 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e323902-93b4-4f7d-8c05-648130742ac1 admin admin] PATH_INFO: `/v3/projects/a14cbb13a8174f0fb62ffe8a1f0ef00f` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:47.684168 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e323902-93b4-4f7d-8c05-648130742ac1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a14cbb13a8174f0fb62ffe8a1f0ef00f)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:47.687050 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e323902-93b4-4f7d-8c05-648130742ac1 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:47.792838 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-0e323902-93b4-4f7d-8c05-648130742ac1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a14cbb13a8174f0fb62ffe8a1f0ef00f', 'request_id': 'req-0e323902-93b4-4f7d-8c05-648130742ac1'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:47.799727 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-0e323902-93b4-4f7d-8c05-648130742ac1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a14cbb13a8174f0fb62ffe8a1f0ef00f', 'request_id': 'req-0e323902-93b4-4f7d-8c05-648130742ac1'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:47.840643 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-0e323902-93b4-4f7d-8c05-648130742ac1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a14cbb13a8174f0fb62ffe8a1f0ef00f', 'request_id': 'req-0e323902-93b4-4f7d-8c05-648130742ac1'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:47.852938 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-0e323902-93b4-4f7d-8c05-648130742ac1 admin admin] The token cache is being invalidate because project a14cbb13a8174f0fb62ffe8a1f0ef00f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 08:42:47.853266 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-0e323902-93b4-4f7d-8c05-648130742ac1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:47.853906 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-0e323902-93b4-4f7d-8c05-648130742ac1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:47.855405 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 950/1889] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 08:42:47 2022] DELETE /identity/v3/projects/a14cbb13a8174f0fb62ffe8a1f0ef00f => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:42:47.859549 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb1712a-ade0-476f-b697-46017f7f1026 service neutron] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:47.869822 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb1712a-ade0-476f-b697-46017f7f1026 service neutron] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:47.928066 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1a7e0f-91ce-4359-a11e-fe7f24d310c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:47.928757 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1a7e0f-91ce-4359-a11e-fe7f24d310c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:47.929074 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1a7e0f-91ce-4359-a11e-fe7f24d310c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:47.929351 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1a7e0f-91ce-4359-a11e-fe7f24d310c9 admin admin] PATH_INFO: `/v3/users/48e76241b6684e85a3737b454b07ec3b` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:47.930472 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1a7e0f-91ce-4359-a11e-fe7f24d310c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=48e76241b6684e85a3737b454b07ec3b)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:47.932390 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1a7e0f-91ce-4359-a11e-fe7f24d310c9 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:48.194234 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-0c1a7e0f-91ce-4359-a11e-fe7f24d310c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48e76241b6684e85a3737b454b07ec3b'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:48.219221 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-0c1a7e0f-91ce-4359-a11e-fe7f24d310c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48e76241b6684e85a3737b454b07ec3b'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:48.228154 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-0c1a7e0f-91ce-4359-a11e-fe7f24d310c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48e76241b6684e85a3737b454b07ec3b'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:48.229018 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-0c1a7e0f-91ce-4359-a11e-fe7f24d310c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48e76241b6684e85a3737b454b07ec3b'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:48.231901 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-0c1a7e0f-91ce-4359-a11e-fe7f24d310c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48e76241b6684e85a3737b454b07ec3b'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:48.339746 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 940/1890] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 08:42:47 2022] DELETE /identity/v3/users/48e76241b6684e85a3737b454b07ec3b => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 08:42:48.911898 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e323902-93b4-4f7d-8c05-648130742ac1 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:48.913455 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-0e323902-93b4-4f7d-8c05-648130742ac1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:42:48.926958 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e323902-93b4-4f7d-8c05-648130742ac1 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:49.000674 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b07e5d1-3a40-485a-81d8-b83ff2ccc65e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:49.001235 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b07e5d1-3a40-485a-81d8-b83ff2ccc65e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:49.001640 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b07e5d1-3a40-485a-81d8-b83ff2ccc65e admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:49.001921 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b07e5d1-3a40-485a-81d8-b83ff2ccc65e admin admin] PATH_INFO: `/v3/projects/058f48df4756449eb52a520960593e09` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:49.003097 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b07e5d1-3a40-485a-81d8-b83ff2ccc65e admin admin] RBAC: Authorizing `identity:delete_project(project_id=058f48df4756449eb52a520960593e09)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:49.005729 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b07e5d1-3a40-485a-81d8-b83ff2ccc65e admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:49.094382 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:42:49.111539 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-7b07e5d1-3a40-485a-81d8-b83ff2ccc65e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '058f48df4756449eb52a520960593e09', 'request_id': 'req-7b07e5d1-3a40-485a-81d8-b83ff2ccc65e'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:49.118001 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-7b07e5d1-3a40-485a-81d8-b83ff2ccc65e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '058f48df4756449eb52a520960593e09', 'request_id': 'req-7b07e5d1-3a40-485a-81d8-b83ff2ccc65e'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:49.195476 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-7b07e5d1-3a40-485a-81d8-b83ff2ccc65e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '058f48df4756449eb52a520960593e09', 'request_id': 'req-7b07e5d1-3a40-485a-81d8-b83ff2ccc65e'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:49.207392 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-7b07e5d1-3a40-485a-81d8-b83ff2ccc65e admin admin] The token cache is being invalidate because project 058f48df4756449eb52a520960593e09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 08:42:49.207906 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-7b07e5d1-3a40-485a-81d8-b83ff2ccc65e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:49.211352 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-7b07e5d1-3a40-485a-81d8-b83ff2ccc65e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:49.211352 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 951/1891] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 08:42:48 2022] DELETE /identity/v3/projects/058f48df4756449eb52a520960593e09 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:42:49.215386 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078657a2-36d0-4b84-8a29-14c786bd9c6e None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:49.215920 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078657a2-36d0-4b84-8a29-14c786bd9c6e None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:49.215920 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078657a2-36d0-4b84-8a29-14c786bd9c6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:49.236137 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-078657a2-36d0-4b84-8a29-14c786bd9c6e None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:42:49.237368 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-078657a2-36d0-4b84-8a29-14c786bd9c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:42:49.247315 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:42:49.296281 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 941/1892] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:42:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:42:49.305074 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6595aab3-e11d-4a8c-9bb8-cd2c21c7044c None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:49.305362 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6595aab3-e11d-4a8c-9bb8-cd2c21c7044c None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:49.305634 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6595aab3-e11d-4a8c-9bb8-cd2c21c7044c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:49.325868 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-6595aab3-e11d-4a8c-9bb8-cd2c21c7044c None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:42:49.327063 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-6595aab3-e11d-4a8c-9bb8-cd2c21c7044c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:42:49.336876 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 952/1893] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:42:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:42:49.340875 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078657a2-36d0-4b84-8a29-14c786bd9c6e None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:49.352851 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078657a2-36d0-4b84-8a29-14c786bd9c6e None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:49.359212 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1f308b-1e4b-431f-8ace-d495adc7c4ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:49.360003 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1f308b-1e4b-431f-8ace-d495adc7c4ce admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:49.360282 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1f308b-1e4b-431f-8ace-d495adc7c4ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:49.360848 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1f308b-1e4b-431f-8ace-d495adc7c4ce admin admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:49.361528 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1f308b-1e4b-431f-8ace-d495adc7c4ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:42:49.361828 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1f308b-1e4b-431f-8ace-d495adc7c4ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:49.363455 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1f308b-1e4b-431f-8ace-d495adc7c4ce admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:49.382125 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6595aab3-e11d-4a8c-9bb8-cd2c21c7044c None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:49.385158 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 942/1894] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Aug 10 08:42:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:42:49.393735 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6595aab3-e11d-4a8c-9bb8-cd2c21c7044c None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:49.401123 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df43b49e-8ee5-4dc6-9631-a01e4ef747e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:49.401803 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df43b49e-8ee5-4dc6-9631-a01e4ef747e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:49.402222 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df43b49e-8ee5-4dc6-9631-a01e4ef747e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:49.402348 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df43b49e-8ee5-4dc6-9631-a01e4ef747e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:49.405008 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df43b49e-8ee5-4dc6-9631-a01e4ef747e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:49.406991 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df43b49e-8ee5-4dc6-9631-a01e4ef747e0 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:49.484207 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 953/1895] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 08:42:49 2022] POST /identity/v3/projects => generated 387 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 08:42:49.488470 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1f308b-1e4b-431f-8ace-d495adc7c4ce admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:49.499056 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1f308b-1e4b-431f-8ace-d495adc7c4ce admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:49.560859 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d19b05-dc4f-40f3-b1ae-0b7d0c2d6103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:49.561507 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d19b05-dc4f-40f3-b1ae-0b7d0c2d6103 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:49.561790 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d19b05-dc4f-40f3-b1ae-0b7d0c2d6103 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:49.562062 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d19b05-dc4f-40f3-b1ae-0b7d0c2d6103 admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:49.562726 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d19b05-dc4f-40f3-b1ae-0b7d0c2d6103 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:49.564643 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d19b05-dc4f-40f3-b1ae-0b7d0c2d6103 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:49.630473 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 943/1896] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 08:42:49 2022] POST /identity/v3/users => generated 294 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 08:42:49.635145 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df43b49e-8ee5-4dc6-9631-a01e4ef747e0 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:49.646669 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df43b49e-8ee5-4dc6-9631-a01e4ef747e0 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:49.651287 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5961f8c-ed4b-4d8a-b5a9-715a39f08726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:49.651893 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5961f8c-ed4b-4d8a-b5a9-715a39f08726 admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:49.652186 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5961f8c-ed4b-4d8a-b5a9-715a39f08726 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:49.652457 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5961f8c-ed4b-4d8a-b5a9-715a39f08726 admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:49.654841 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5961f8c-ed4b-4d8a-b5a9-715a39f08726 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:49.654841 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5961f8c-ed4b-4d8a-b5a9-715a39f08726 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:49.669611 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 954/1897] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:42:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 08:42:49.670817 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d19b05-dc4f-40f3-b1ae-0b7d0c2d6103 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:49.681205 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d19b05-dc4f-40f3-b1ae-0b7d0c2d6103 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:49.685948 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ac8de2-ce44-47ab-a3cb-f45ceb40eed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:49.686555 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ac8de2-ce44-47ab-a3cb-f45ceb40eed0 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:49.686835 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ac8de2-ce44-47ab-a3cb-f45ceb40eed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:49.687115 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ac8de2-ce44-47ab-a3cb-f45ceb40eed0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:49.687762 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ac8de2-ce44-47ab-a3cb-f45ceb40eed0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:49.689442 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ac8de2-ce44-47ab-a3cb-f45ceb40eed0 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:49.703225 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 944/1898] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:42:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:42:49.704837 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5961f8c-ed4b-4d8a-b5a9-715a39f08726 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:49.720547 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5961f8c-ed4b-4d8a-b5a9-715a39f08726 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:49.731999 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f200ab7-6a61-4c8f-a4d4-a887b9217b2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:49.733316 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f200ab7-6a61-4c8f-a4d4-a887b9217b2f admin admin] REQUEST_METHOD: `PUT` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:49.733769 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f200ab7-6a61-4c8f-a4d4-a887b9217b2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:49.734324 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f200ab7-6a61-4c8f-a4d4-a887b9217b2f admin admin] PATH_INFO: `/v3/projects/867b600e79d040ccb23dfd44cd36c0cc/users/0ae16edf81394419895607ce058add00/roles/44b64db484f24a46bf0b7ced46b519bf` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:49.758686 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f200ab7-6a61-4c8f-a4d4-a887b9217b2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=867b600e79d040ccb23dfd44cd36c0cc, user_id=0ae16edf81394419895607ce058add00, role_id=44b64db484f24a46bf0b7ced46b519bf)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:49.761501 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f200ab7-6a61-4c8f-a4d4-a887b9217b2f admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:49.846772 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 955/1899] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 08:42:49 2022] PUT /identity/v3/projects/867b600e79d040ccb23dfd44cd36c0cc/users/0ae16edf81394419895607ce058add00/roles/44b64db484f24a46bf0b7ced46b519bf => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:42:49.858361 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b01de1-2a30-496b-a24f-8b9a33800a1a None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:49.858647 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b01de1-2a30-496b-a24f-8b9a33800a1a None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:49.858940 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b01de1-2a30-496b-a24f-8b9a33800a1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:49.886788 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-95b01de1-2a30-496b-a24f-8b9a33800a1a None None] MFA Rules not processed for user `0ae16edf81394419895607ce058add00`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:42:49.887991 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-95b01de1-2a30-496b-a24f-8b9a33800a1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:42:49.950931 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 945/1900] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:42:49 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:42:49.955429 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f200ab7-6a61-4c8f-a4d4-a887b9217b2f admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:50.016824 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f200ab7-6a61-4c8f-a4d4-a887b9217b2f admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:50.022450 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6762fc3b-2974-47cd-a585-5aa17a307c23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:50.023053 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6762fc3b-2974-47cd-a585-5aa17a307c23 admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:50.023330 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6762fc3b-2974-47cd-a585-5aa17a307c23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:50.023598 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6762fc3b-2974-47cd-a585-5aa17a307c23 admin admin] PATH_INFO: `/v3/projects` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:50.024221 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6762fc3b-2974-47cd-a585-5aa17a307c23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:50.026149 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6762fc3b-2974-47cd-a585-5aa17a307c23 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:50.132556 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 956/1901] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 08:42:49 2022] POST /identity/v3/projects => generated 385 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:42:50.133715 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b01de1-2a30-496b-a24f-8b9a33800a1a None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:50.144542 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b01de1-2a30-496b-a24f-8b9a33800a1a None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:50.154113 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:42:50.204695 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2569e3e-3231-4bf4-b3f8-014047936b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:50.205341 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2569e3e-3231-4bf4-b3f8-014047936b06 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:50.205624 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2569e3e-3231-4bf4-b3f8-014047936b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:50.205892 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2569e3e-3231-4bf4-b3f8-014047936b06 admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:50.206517 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2569e3e-3231-4bf4-b3f8-014047936b06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:50.208495 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2569e3e-3231-4bf4-b3f8-014047936b06 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:50.267691 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 946/1902] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 08:42:50 2022] POST /identity/v3/users => generated 293 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 08:42:50.269710 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6762fc3b-2974-47cd-a585-5aa17a307c23 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:50.281148 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6762fc3b-2974-47cd-a585-5aa17a307c23 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:50.286376 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2b5e11-a56e-4e85-ad13-c09e99fe3a6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:50.287038 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2b5e11-a56e-4e85-ad13-c09e99fe3a6d admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:50.287353 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2b5e11-a56e-4e85-ad13-c09e99fe3a6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:50.287630 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2b5e11-a56e-4e85-ad13-c09e99fe3a6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:50.288286 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2b5e11-a56e-4e85-ad13-c09e99fe3a6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:50.289946 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2b5e11-a56e-4e85-ad13-c09e99fe3a6d admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:50.301791 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 957/1903] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:42:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:42:50.306611 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2569e3e-3231-4bf4-b3f8-014047936b06 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:50.317910 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2569e3e-3231-4bf4-b3f8-014047936b06 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:50.323179 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8432745-a2ba-4623-8a1e-25d57f29b39d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:50.323824 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8432745-a2ba-4623-8a1e-25d57f29b39d admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:50.324284 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8432745-a2ba-4623-8a1e-25d57f29b39d admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:50.324391 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8432745-a2ba-4623-8a1e-25d57f29b39d admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:50.325052 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8432745-a2ba-4623-8a1e-25d57f29b39d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:50.326794 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8432745-a2ba-4623-8a1e-25d57f29b39d admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:50.340099 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 947/1904] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:42:50 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:42:50.344314 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2b5e11-a56e-4e85-ad13-c09e99fe3a6d admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:50.354843 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2b5e11-a56e-4e85-ad13-c09e99fe3a6d admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:50.359528 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec67d19e-9ffd-4ce7-917b-11ef2aede9a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:50.360202 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec67d19e-9ffd-4ce7-917b-11ef2aede9a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:50.360481 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec67d19e-9ffd-4ce7-917b-11ef2aede9a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:50.360755 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec67d19e-9ffd-4ce7-917b-11ef2aede9a1 admin admin] PATH_INFO: `/v3/projects/7c90d44540f0458e95a2e5c3c2f25840/users/ad6f1cbb718f450ca0809917bf2af7cf/roles/44b64db484f24a46bf0b7ced46b519bf` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:50.377218 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec67d19e-9ffd-4ce7-917b-11ef2aede9a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c90d44540f0458e95a2e5c3c2f25840, user_id=ad6f1cbb718f450ca0809917bf2af7cf, role_id=44b64db484f24a46bf0b7ced46b519bf)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:50.380623 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec67d19e-9ffd-4ce7-917b-11ef2aede9a1 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:50.444066 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:42:50.456969 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 958/1905] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 08:42:50 2022] PUT /identity/v3/projects/7c90d44540f0458e95a2e5c3c2f25840/users/ad6f1cbb718f450ca0809917bf2af7cf/roles/44b64db484f24a46bf0b7ced46b519bf => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:42:50.465557 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e66852-7a9c-4734-bb25-c28aa5d7bed4 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:50.465875 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e66852-7a9c-4734-bb25-c28aa5d7bed4 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:50.466154 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e66852-7a9c-4734-bb25-c28aa5d7bed4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:50.493910 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-14e66852-7a9c-4734-bb25-c28aa5d7bed4 None None] MFA Rules not processed for user `ad6f1cbb718f450ca0809917bf2af7cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:42:50.495095 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-14e66852-7a9c-4734-bb25-c28aa5d7bed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:42:50.559132 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 948/1906] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:42:50 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:42:50.563482 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec67d19e-9ffd-4ce7-917b-11ef2aede9a1 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:50.631687 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec67d19e-9ffd-4ce7-917b-11ef2aede9a1 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:50.636300 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f6e1e0-0475-4b79-9377-c57f3e345780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:50.636936 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f6e1e0-0475-4b79-9377-c57f3e345780 admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:50.639763 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f6e1e0-0475-4b79-9377-c57f3e345780 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:50.639763 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f6e1e0-0475-4b79-9377-c57f3e345780 admin admin] PATH_INFO: `/v3/projects` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:50.639763 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f6e1e0-0475-4b79-9377-c57f3e345780 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:50.640007 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f6e1e0-0475-4b79-9377-c57f3e345780 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:50.723938 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 959/1907] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 08:42:50 2022] POST /identity/v3/projects => generated 387 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:42:50.727889 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e66852-7a9c-4734-bb25-c28aa5d7bed4 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:50.739781 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e66852-7a9c-4734-bb25-c28aa5d7bed4 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:50.803657 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2429139-ea7c-46f9-90fd-f0e963bff45a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:50.804353 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2429139-ea7c-46f9-90fd-f0e963bff45a admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:50.804692 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2429139-ea7c-46f9-90fd-f0e963bff45a admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:50.805070 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2429139-ea7c-46f9-90fd-f0e963bff45a admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:50.805793 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2429139-ea7c-46f9-90fd-f0e963bff45a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:50.807933 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2429139-ea7c-46f9-90fd-f0e963bff45a admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:50.882186 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 949/1908] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 08:42:50 2022] POST /identity/v3/users => generated 300 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:42:50.886474 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f6e1e0-0475-4b79-9377-c57f3e345780 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:50.897434 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f6e1e0-0475-4b79-9377-c57f3e345780 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:50.902013 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e98d75-127b-40e2-b757-b206f38fac74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:50.902678 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e98d75-127b-40e2-b757-b206f38fac74 admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:50.902961 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e98d75-127b-40e2-b757-b206f38fac74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:50.903233 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e98d75-127b-40e2-b757-b206f38fac74 admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:50.903902 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e98d75-127b-40e2-b757-b206f38fac74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:50.905610 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e98d75-127b-40e2-b757-b206f38fac74 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:50.917087 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 960/1909] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:42:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:42:50.921390 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2429139-ea7c-46f9-90fd-f0e963bff45a admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:50.932900 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2429139-ea7c-46f9-90fd-f0e963bff45a admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:50.937780 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24c880f-35bd-43d5-a244-1a6478485885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:50.938519 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24c880f-35bd-43d5-a244-1a6478485885 admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:50.938810 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24c880f-35bd-43d5-a244-1a6478485885 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:50.939078 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24c880f-35bd-43d5-a244-1a6478485885 admin admin] PATH_INFO: `/v3/projects/fd8c6e2240a94ac8b156236aecaf3b88/users/32e2909bf5814108a1c20b4717477654/roles/ec3751692d9c4ecf81b4a71c936c46be` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:50.942834 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:42:50.955009 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24c880f-35bd-43d5-a244-1a6478485885 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd8c6e2240a94ac8b156236aecaf3b88, user_id=32e2909bf5814108a1c20b4717477654, role_id=ec3751692d9c4ecf81b4a71c936c46be)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:50.958284 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24c880f-35bd-43d5-a244-1a6478485885 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:51.030094 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 950/1910] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 08:42:50 2022] PUT /identity/v3/projects/fd8c6e2240a94ac8b156236aecaf3b88/users/32e2909bf5814108a1c20b4717477654/roles/ec3751692d9c4ecf81b4a71c936c46be => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 08:42:51.034068 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e98d75-127b-40e2-b757-b206f38fac74 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:51.044473 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e98d75-127b-40e2-b757-b206f38fac74 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:51.100096 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5806297a-4316-436f-a85a-476a68e477ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:51.100693 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5806297a-4316-436f-a85a-476a68e477ab admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:51.101001 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5806297a-4316-436f-a85a-476a68e477ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:51.101271 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5806297a-4316-436f-a85a-476a68e477ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:51.101897 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5806297a-4316-436f-a85a-476a68e477ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:51.103474 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5806297a-4316-436f-a85a-476a68e477ab admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:51.113051 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 961/1911] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:42:51 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:42:51.117855 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24c880f-35bd-43d5-a244-1a6478485885 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:51.128420 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24c880f-35bd-43d5-a244-1a6478485885 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:51.134427 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9b62cb-311c-4bf3-9e1b-5b11338b7936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:51.135116 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9b62cb-311c-4bf3-9e1b-5b11338b7936 admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:51.135396 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9b62cb-311c-4bf3-9e1b-5b11338b7936 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:51.135665 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9b62cb-311c-4bf3-9e1b-5b11338b7936 admin admin] PATH_INFO: `/v3/projects/fd8c6e2240a94ac8b156236aecaf3b88/users/32e2909bf5814108a1c20b4717477654/roles/ec3751692d9c4ecf81b4a71c936c46be` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:51.137573 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9b62cb-311c-4bf3-9e1b-5b11338b7936 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd8c6e2240a94ac8b156236aecaf3b88, user_id=32e2909bf5814108a1c20b4717477654, role_id=ec3751692d9c4ecf81b4a71c936c46be)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:51.140197 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9b62cb-311c-4bf3-9e1b-5b11338b7936 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:51.150614 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 951/1912] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 08:42:51 2022] PUT /identity/v3/projects/fd8c6e2240a94ac8b156236aecaf3b88/users/32e2909bf5814108a1c20b4717477654/roles/ec3751692d9c4ecf81b4a71c936c46be => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:42:51.160330 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4729f3-621f-4cad-a687-02a46d5f9f6c None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:51.160618 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4729f3-621f-4cad-a687-02a46d5f9f6c None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:51.160888 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4729f3-621f-4cad-a687-02a46d5f9f6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:51.188015 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-8d4729f3-621f-4cad-a687-02a46d5f9f6c None None] MFA Rules not processed for user `32e2909bf5814108a1c20b4717477654`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:42:51.189260 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-8d4729f3-621f-4cad-a687-02a46d5f9f6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:42:51.246589 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:42:51.256074 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 962/1913] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:42:51 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:42:51.266662 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9b62cb-311c-4bf3-9e1b-5b11338b7936 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:51.268297 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-1d9b62cb-311c-4bf3-9e1b-5b11338b7936 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:42:51.325577 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9b62cb-311c-4bf3-9e1b-5b11338b7936 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:51.329363 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cc74b8-2f48-43ec-8f97-d0bcb69072a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ba74e28ced4cbba857f54222dfbe41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:51.329963 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cc74b8-2f48-43ec-8f97-d0bcb69072a3 service cinder] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:51.330237 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cc74b8-2f48-43ec-8f97-d0bcb69072a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:51.330505 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cc74b8-2f48-43ec-8f97-d0bcb69072a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:51.339410 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cc74b8-2f48-43ec-8f97-d0bcb69072a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:51.340832 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cc74b8-2f48-43ec-8f97-d0bcb69072a3 service cinder] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:51.350882 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 952/1914] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:42:51 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:42:52.445234 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4729f3-621f-4cad-a687-02a46d5f9f6c None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:52.451637 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-8d4729f3-621f-4cad-a687-02a46d5f9f6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:42:52.458816 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4729f3-621f-4cad-a687-02a46d5f9f6c None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:52.462411 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd4f76c-c664-4bad-904d-80396c25d2c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ba74e28ced4cbba857f54222dfbe41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:52.463001 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd4f76c-c664-4bad-904d-80396c25d2c1 service cinder] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:52.463280 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd4f76c-c664-4bad-904d-80396c25d2c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:52.463554 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd4f76c-c664-4bad-904d-80396c25d2c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:52.519859 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd4f76c-c664-4bad-904d-80396c25d2c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:52.522797 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd4f76c-c664-4bad-904d-80396c25d2c1 service cinder] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:52.532458 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 963/1915] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:42:52 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:42:54.011579 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cc74b8-2f48-43ec-8f97-d0bcb69072a3 service cinder] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:54.013064 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-73cc74b8-2f48-43ec-8f97-d0bcb69072a3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:42:54.084981 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cc74b8-2f48-43ec-8f97-d0bcb69072a3 service cinder] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:54.090472 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed0f2cf-92fe-4b78-834c-a7d0db909dc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:54.091116 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed0f2cf-92fe-4b78-834c-a7d0db909dc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:54.092551 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed0f2cf-92fe-4b78-834c-a7d0db909dc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:54.092551 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed0f2cf-92fe-4b78-834c-a7d0db909dc5 admin admin] PATH_INFO: `/v3/users/afc19691612448889ba6f41d666a98a0` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:54.092810 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed0f2cf-92fe-4b78-834c-a7d0db909dc5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=afc19691612448889ba6f41d666a98a0)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:54.094988 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed0f2cf-92fe-4b78-834c-a7d0db909dc5 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:54.318027 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-aed0f2cf-92fe-4b78-834c-a7d0db909dc5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afc19691612448889ba6f41d666a98a0'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:54.329897 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-aed0f2cf-92fe-4b78-834c-a7d0db909dc5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afc19691612448889ba6f41d666a98a0'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:54.337039 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-aed0f2cf-92fe-4b78-834c-a7d0db909dc5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afc19691612448889ba6f41d666a98a0'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:54.337715 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-aed0f2cf-92fe-4b78-834c-a7d0db909dc5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afc19691612448889ba6f41d666a98a0'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:54.342578 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-aed0f2cf-92fe-4b78-834c-a7d0db909dc5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afc19691612448889ba6f41d666a98a0'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:54.342578 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:42:54.452728 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 953/1916] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 08:42:54 2022] DELETE /identity/v3/users/afc19691612448889ba6f41d666a98a0 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:42:54.457491 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b08640-8aa5-492a-a13f-b09190cd5456 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:54.461175 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b08640-8aa5-492a-a13f-b09190cd5456 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:54.461175 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b08640-8aa5-492a-a13f-b09190cd5456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:54.480537 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-63b08640-8aa5-492a-a13f-b09190cd5456 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:42:54.492447 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-63b08640-8aa5-492a-a13f-b09190cd5456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:42:54.559862 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 964/1917] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:42:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:42:54.571510 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed0f2cf-92fe-4b78-834c-a7d0db909dc5 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:54.573137 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-aed0f2cf-92fe-4b78-834c-a7d0db909dc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:42:54.634063 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed0f2cf-92fe-4b78-834c-a7d0db909dc5 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:54.637723 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d489487f-d8d4-466c-ba9f-e070495e1dbe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:54.638333 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d489487f-d8d4-466c-ba9f-e070495e1dbe service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:54.638610 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d489487f-d8d4-466c-ba9f-e070495e1dbe service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:54.638881 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d489487f-d8d4-466c-ba9f-e070495e1dbe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:54.648808 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d489487f-d8d4-466c-ba9f-e070495e1dbe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:54.650280 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d489487f-d8d4-466c-ba9f-e070495e1dbe service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:54.660325 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 954/1918] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:42:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 08:42:55.251921 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b08640-8aa5-492a-a13f-b09190cd5456 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:55.253852 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-63b08640-8aa5-492a-a13f-b09190cd5456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:42:55.269454 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b08640-8aa5-492a-a13f-b09190cd5456 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:55.274797 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddc38e3-1cf9-4bad-ae6c-105ddd374af9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:55.275454 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddc38e3-1cf9-4bad-ae6c-105ddd374af9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:55.275737 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddc38e3-1cf9-4bad-ae6c-105ddd374af9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:55.276154 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddc38e3-1cf9-4bad-ae6c-105ddd374af9 admin admin] PATH_INFO: `/v3/projects/c547b4ce9248430a9fbccf818c7203b3` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:55.277658 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddc38e3-1cf9-4bad-ae6c-105ddd374af9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c547b4ce9248430a9fbccf818c7203b3)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:55.280547 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddc38e3-1cf9-4bad-ae6c-105ddd374af9 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:55.438768 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-3ddc38e3-1cf9-4bad-ae6c-105ddd374af9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c547b4ce9248430a9fbccf818c7203b3', 'request_id': 'req-3ddc38e3-1cf9-4bad-ae6c-105ddd374af9'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:55.445629 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-3ddc38e3-1cf9-4bad-ae6c-105ddd374af9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c547b4ce9248430a9fbccf818c7203b3', 'request_id': 'req-3ddc38e3-1cf9-4bad-ae6c-105ddd374af9'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:55.550941 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-3ddc38e3-1cf9-4bad-ae6c-105ddd374af9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c547b4ce9248430a9fbccf818c7203b3', 'request_id': 'req-3ddc38e3-1cf9-4bad-ae6c-105ddd374af9'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:55.565686 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-3ddc38e3-1cf9-4bad-ae6c-105ddd374af9 admin admin] The token cache is being invalidate because project c547b4ce9248430a9fbccf818c7203b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 08:42:55.566012 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-3ddc38e3-1cf9-4bad-ae6c-105ddd374af9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:55.566687 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-3ddc38e3-1cf9-4bad-ae6c-105ddd374af9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:55.568942 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 965/1919] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 08:42:55 2022] DELETE /identity/v3/projects/c547b4ce9248430a9fbccf818c7203b3 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:42:55.572753 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d489487f-d8d4-466c-ba9f-e070495e1dbe service neutron] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:55.583364 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d489487f-d8d4-466c-ba9f-e070495e1dbe service neutron] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:55.677016 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14a8b00-5369-4e50-82f6-3b8d08784c10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:55.677777 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14a8b00-5369-4e50-82f6-3b8d08784c10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:55.678176 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14a8b00-5369-4e50-82f6-3b8d08784c10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:55.678673 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14a8b00-5369-4e50-82f6-3b8d08784c10 admin admin] PATH_INFO: `/v3/users/2e6de0376fb34cd981b3a4c485f25a8d` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:55.679823 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14a8b00-5369-4e50-82f6-3b8d08784c10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e6de0376fb34cd981b3a4c485f25a8d)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:55.681976 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14a8b00-5369-4e50-82f6-3b8d08784c10 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:55.925360 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-a14a8b00-5369-4e50-82f6-3b8d08784c10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e6de0376fb34cd981b3a4c485f25a8d'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:55.952565 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-a14a8b00-5369-4e50-82f6-3b8d08784c10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e6de0376fb34cd981b3a4c485f25a8d'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:55.964407 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-a14a8b00-5369-4e50-82f6-3b8d08784c10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e6de0376fb34cd981b3a4c485f25a8d'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:55.965506 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-a14a8b00-5369-4e50-82f6-3b8d08784c10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e6de0376fb34cd981b3a4c485f25a8d'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:55.967368 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-a14a8b00-5369-4e50-82f6-3b8d08784c10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e6de0376fb34cd981b3a4c485f25a8d'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:56.055499 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 955/1920] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 08:42:55 2022] DELETE /identity/v3/users/2e6de0376fb34cd981b3a4c485f25a8d => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:42:56.516581 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddc38e3-1cf9-4bad-ae6c-105ddd374af9 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:56.518189 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-3ddc38e3-1cf9-4bad-ae6c-105ddd374af9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:42:56.543469 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddc38e3-1cf9-4bad-ae6c-105ddd374af9 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:56.560443 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:42:56.609937 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94eb7b5-4428-4a1f-8e5e-fe5060b64fb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:56.610839 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94eb7b5-4428-4a1f-8e5e-fe5060b64fb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:56.611195 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94eb7b5-4428-4a1f-8e5e-fe5060b64fb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:56.611465 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94eb7b5-4428-4a1f-8e5e-fe5060b64fb3 admin admin] PATH_INFO: `/v3/projects/c9ea639f8d1543b9b884cff530230ea8` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:56.612654 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94eb7b5-4428-4a1f-8e5e-fe5060b64fb3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9ea639f8d1543b9b884cff530230ea8)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:56.615381 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94eb7b5-4428-4a1f-8e5e-fe5060b64fb3 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:56.733045 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-a94eb7b5-4428-4a1f-8e5e-fe5060b64fb3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9ea639f8d1543b9b884cff530230ea8', 'request_id': 'req-a94eb7b5-4428-4a1f-8e5e-fe5060b64fb3'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:56.738467 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-a94eb7b5-4428-4a1f-8e5e-fe5060b64fb3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9ea639f8d1543b9b884cff530230ea8', 'request_id': 'req-a94eb7b5-4428-4a1f-8e5e-fe5060b64fb3'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:56.817579 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-a94eb7b5-4428-4a1f-8e5e-fe5060b64fb3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9ea639f8d1543b9b884cff530230ea8', 'request_id': 'req-a94eb7b5-4428-4a1f-8e5e-fe5060b64fb3'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:56.829694 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-a94eb7b5-4428-4a1f-8e5e-fe5060b64fb3 admin admin] The token cache is being invalidate because project c9ea639f8d1543b9b884cff530230ea8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 08:42:56.830182 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-a94eb7b5-4428-4a1f-8e5e-fe5060b64fb3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:56.830978 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-a94eb7b5-4428-4a1f-8e5e-fe5060b64fb3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:56.833001 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 966/1921] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 08:42:56 2022] DELETE /identity/v3/projects/c9ea639f8d1543b9b884cff530230ea8 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:42:56.837190 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14a8b00-5369-4e50-82f6-3b8d08784c10 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:56.838888 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-a14a8b00-5369-4e50-82f6-3b8d08784c10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:42:56.853170 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14a8b00-5369-4e50-82f6-3b8d08784c10 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:56.886228 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:42:56.959197 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee78161f-f935-493f-aa7a-a9b3389eec40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:56.959897 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee78161f-f935-493f-aa7a-a9b3389eec40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:56.960602 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee78161f-f935-493f-aa7a-a9b3389eec40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:56.960876 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee78161f-f935-493f-aa7a-a9b3389eec40 admin admin] PATH_INFO: `/v3/users/865db966e32e44cba734f7ff743006d4` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:56.963784 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee78161f-f935-493f-aa7a-a9b3389eec40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=865db966e32e44cba734f7ff743006d4)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:56.966134 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee78161f-f935-493f-aa7a-a9b3389eec40 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:57.183399 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-ee78161f-f935-493f-aa7a-a9b3389eec40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '865db966e32e44cba734f7ff743006d4'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:57.206369 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-ee78161f-f935-493f-aa7a-a9b3389eec40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '865db966e32e44cba734f7ff743006d4'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:57.210901 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-ee78161f-f935-493f-aa7a-a9b3389eec40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '865db966e32e44cba734f7ff743006d4'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:57.211622 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-ee78161f-f935-493f-aa7a-a9b3389eec40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '865db966e32e44cba734f7ff743006d4'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:57.212241 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-ee78161f-f935-493f-aa7a-a9b3389eec40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '865db966e32e44cba734f7ff743006d4'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:57.288123 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 956/1922] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 08:42:56 2022] DELETE /identity/v3/users/865db966e32e44cba734f7ff743006d4 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:42:57.623687 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94eb7b5-4428-4a1f-8e5e-fe5060b64fb3 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:57.625197 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-a94eb7b5-4428-4a1f-8e5e-fe5060b64fb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:42:57.696319 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94eb7b5-4428-4a1f-8e5e-fe5060b64fb3 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:57.751025 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f509f7d-4601-444d-a712-6ffc536ebf4a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ba74e28ced4cbba857f54222dfbe41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:57.751629 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f509f7d-4601-444d-a712-6ffc536ebf4a service cinder] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:57.751903 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f509f7d-4601-444d-a712-6ffc536ebf4a service cinder] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:57.752170 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f509f7d-4601-444d-a712-6ffc536ebf4a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:57.753879 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-4f509f7d-4601-444d-a712-6ffc536ebf4a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:42:57.772411 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:42:57.774787 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee78161f-f935-493f-aa7a-a9b3389eec40 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:57.777961 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-ee78161f-f935-493f-aa7a-a9b3389eec40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:42:57.836468 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f509f7d-4601-444d-a712-6ffc536ebf4a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:57.838791 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f509f7d-4601-444d-a712-6ffc536ebf4a service cinder] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:57.850000 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 967/1923] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:42:57 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:42:57.874283 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee78161f-f935-493f-aa7a-a9b3389eec40 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:57.883144 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae39457-ac8e-43b6-a2de-818fb3c2f998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:57.883793 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae39457-ac8e-43b6-a2de-818fb3c2f998 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:57.884074 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae39457-ac8e-43b6-a2de-818fb3c2f998 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:57.884346 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae39457-ac8e-43b6-a2de-818fb3c2f998 admin admin] PATH_INFO: `/v3/projects/ef5f4330a7a348c99c6576f4762acc3c` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:57.885630 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae39457-ac8e-43b6-a2de-818fb3c2f998 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef5f4330a7a348c99c6576f4762acc3c)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:57.887603 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae39457-ac8e-43b6-a2de-818fb3c2f998 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:58.011045 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:42:58.019230 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-eae39457-ac8e-43b6-a2de-818fb3c2f998 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef5f4330a7a348c99c6576f4762acc3c', 'request_id': 'req-eae39457-ac8e-43b6-a2de-818fb3c2f998'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:58.097727 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-eae39457-ac8e-43b6-a2de-818fb3c2f998 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef5f4330a7a348c99c6576f4762acc3c', 'request_id': 'req-eae39457-ac8e-43b6-a2de-818fb3c2f998'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:58.103976 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-eae39457-ac8e-43b6-a2de-818fb3c2f998 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef5f4330a7a348c99c6576f4762acc3c', 'request_id': 'req-eae39457-ac8e-43b6-a2de-818fb3c2f998'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:58.115289 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-eae39457-ac8e-43b6-a2de-818fb3c2f998 admin admin] The token cache is being invalidate because project ef5f4330a7a348c99c6576f4762acc3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 08:42:58.115605 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-eae39457-ac8e-43b6-a2de-818fb3c2f998 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:58.116252 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-eae39457-ac8e-43b6-a2de-818fb3c2f998 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:58.117805 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 957/1924] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 08:42:57 2022] DELETE /identity/v3/projects/ef5f4330a7a348c99c6576f4762acc3c => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:42:58.124091 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b2ef98-e675-411f-a900-2ff688b2ba9c None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:58.124379 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b2ef98-e675-411f-a900-2ff688b2ba9c None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:58.124654 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b2ef98-e675-411f-a900-2ff688b2ba9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:58.144128 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-f5b2ef98-e675-411f-a900-2ff688b2ba9c None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:42:58.146174 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-f5b2ef98-e675-411f-a900-2ff688b2ba9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:42:58.246316 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 968/1925] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:42:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:42:58.254665 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4c712c-ef33-499b-a2dc-e3c6b25e1c89 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:58.255050 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4c712c-ef33-499b-a2dc-e3c6b25e1c89 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:58.255344 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4c712c-ef33-499b-a2dc-e3c6b25e1c89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:58.278798 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-fd4c712c-ef33-499b-a2dc-e3c6b25e1c89 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:42:58.280006 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-fd4c712c-ef33-499b-a2dc-e3c6b25e1c89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:42:58.289428 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b2ef98-e675-411f-a900-2ff688b2ba9c None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:58.289802 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 958/1926] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:42:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:42:58.292371 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-f5b2ef98-e675-411f-a900-2ff688b2ba9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:42:58.293944 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4c712c-ef33-499b-a2dc-e3c6b25e1c89 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:58.310467 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b2ef98-e675-411f-a900-2ff688b2ba9c None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:58.312205 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4c712c-ef33-499b-a2dc-e3c6b25e1c89 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:58.323605 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3714d586-adbd-4bba-8d61-44957c089ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:58.323605 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3714d586-adbd-4bba-8d61-44957c089ca7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:58.323605 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3714d586-adbd-4bba-8d61-44957c089ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:58.323605 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3714d586-adbd-4bba-8d61-44957c089ca7 admin admin] PATH_INFO: `/v3/users/e07a2f0f9d1f4bdb82e3c8cd87cba512` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:58.323996 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3714d586-adbd-4bba-8d61-44957c089ca7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e07a2f0f9d1f4bdb82e3c8cd87cba512)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:58.323996 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3714d586-adbd-4bba-8d61-44957c089ca7 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:58.324855 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f756726-9561-4b72-819a-034c163377e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:58.325497 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f756726-9561-4b72-819a-034c163377e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:58.325773 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f756726-9561-4b72-819a-034c163377e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:58.326092 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f756726-9561-4b72-819a-034c163377e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:58.326688 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f756726-9561-4b72-819a-034c163377e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:42:58.326989 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f756726-9561-4b72-819a-034c163377e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:58.328583 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f756726-9561-4b72-819a-034c163377e0 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:58.361096 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 959/1927] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Aug 10 08:42:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:42:58.363910 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f756726-9561-4b72-819a-034c163377e0 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:58.377012 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f756726-9561-4b72-819a-034c163377e0 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:58.383103 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d33702-2fc9-411f-8ce4-1442f120f9b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:58.385594 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d33702-2fc9-411f-8ce4-1442f120f9b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:58.385594 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d33702-2fc9-411f-8ce4-1442f120f9b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:58.385594 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d33702-2fc9-411f-8ce4-1442f120f9b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:58.385594 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d33702-2fc9-411f-8ce4-1442f120f9b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:58.387337 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d33702-2fc9-411f-8ce4-1442f120f9b6 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:58.488289 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 960/1928] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 08:42:58 2022] POST /identity/v3/projects => generated 395 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:42:58.492989 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d33702-2fc9-411f-8ce4-1442f120f9b6 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:58.543948 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-3714d586-adbd-4bba-8d61-44957c089ca7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e07a2f0f9d1f4bdb82e3c8cd87cba512'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:58.560357 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-3714d586-adbd-4bba-8d61-44957c089ca7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e07a2f0f9d1f4bdb82e3c8cd87cba512'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:58.569025 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d33702-2fc9-411f-8ce4-1442f120f9b6 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:58.570217 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-3714d586-adbd-4bba-8d61-44957c089ca7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e07a2f0f9d1f4bdb82e3c8cd87cba512'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:58.574220 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c228dde0-53d7-4be9-a9b6-13187471c4b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:58.574764 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c228dde0-53d7-4be9-a9b6-13187471c4b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:58.575044 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c228dde0-53d7-4be9-a9b6-13187471c4b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:58.575313 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c228dde0-53d7-4be9-a9b6-13187471c4b1 admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:58.575985 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c228dde0-53d7-4be9-a9b6-13187471c4b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:58.577973 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c228dde0-53d7-4be9-a9b6-13187471c4b1 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:58.624008 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-3714d586-adbd-4bba-8d61-44957c089ca7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e07a2f0f9d1f4bdb82e3c8cd87cba512'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:58.624651 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-3714d586-adbd-4bba-8d61-44957c089ca7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e07a2f0f9d1f4bdb82e3c8cd87cba512'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:58.632231 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 969/1929] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 08:42:58 2022] DELETE /identity/v3/users/e07a2f0f9d1f4bdb82e3c8cd87cba512 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 08:42:58.636463 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531c4db8-4f33-4f91-a601-55b9418c487b None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:58.636747 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531c4db8-4f33-4f91-a601-55b9418c487b None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:58.637067 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531c4db8-4f33-4f91-a601-55b9418c487b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:58.654698 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 961/1930] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 08:42:58 2022] POST /identity/v3/users => generated 298 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 08:42:58.658693 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c228dde0-53d7-4be9-a9b6-13187471c4b1 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:58.659016 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-531c4db8-4f33-4f91-a601-55b9418c487b None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:42:58.660120 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-c228dde0-53d7-4be9-a9b6-13187471c4b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:42:58.660322 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-531c4db8-4f33-4f91-a601-55b9418c487b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:42:58.673707 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c228dde0-53d7-4be9-a9b6-13187471c4b1 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:58.687810 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:42:58.735096 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 970/1931] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:42:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 08:42:58.745595 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531c4db8-4f33-4f91-a601-55b9418c487b None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:58.747054 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-531c4db8-4f33-4f91-a601-55b9418c487b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:42:58.751930 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80cabd8-d4d7-4e39-8fca-fb1ca9d030d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:58.752554 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80cabd8-d4d7-4e39-8fca-fb1ca9d030d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:58.752836 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80cabd8-d4d7-4e39-8fca-fb1ca9d030d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:58.753141 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80cabd8-d4d7-4e39-8fca-fb1ca9d030d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:58.753804 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80cabd8-d4d7-4e39-8fca-fb1ca9d030d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:58.757225 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80cabd8-d4d7-4e39-8fca-fb1ca9d030d9 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:58.780629 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 962/1932] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:42:58 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:42:58.782210 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80cabd8-d4d7-4e39-8fca-fb1ca9d030d9 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:58.796364 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80cabd8-d4d7-4e39-8fca-fb1ca9d030d9 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:58.802868 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a75f68-8127-42fe-a979-109ab2a7a88e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:58.803588 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a75f68-8127-42fe-a979-109ab2a7a88e admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:58.803896 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a75f68-8127-42fe-a979-109ab2a7a88e admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:58.804205 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a75f68-8127-42fe-a979-109ab2a7a88e admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:58.804863 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a75f68-8127-42fe-a979-109ab2a7a88e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:58.806576 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a75f68-8127-42fe-a979-109ab2a7a88e admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:58.809077 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:42:58.821408 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 963/1933] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:42:58 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:42:58.825954 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a75f68-8127-42fe-a979-109ab2a7a88e admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:58.838243 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531c4db8-4f33-4f91-a601-55b9418c487b None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:58.838926 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a75f68-8127-42fe-a979-109ab2a7a88e admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:58.842141 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c925488-8fe3-4b60-8a80-4f7c11473f99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:58.843019 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c925488-8fe3-4b60-8a80-4f7c11473f99 service neutron] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:58.843019 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c925488-8fe3-4b60-8a80-4f7c11473f99 service neutron] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:58.843403 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c925488-8fe3-4b60-8a80-4f7c11473f99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:58.844226 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b083fb-97cb-44a6-ab98-2473a6d64155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:58.845008 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b083fb-97cb-44a6-ab98-2473a6d64155 admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:58.845386 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b083fb-97cb-44a6-ab98-2473a6d64155 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:58.845865 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b083fb-97cb-44a6-ab98-2473a6d64155 admin admin] PATH_INFO: `/v3/projects/1ec81b012136413ba8fb9a9f61ca6625/users/534a012969ef4493868aee2737da0a74/roles/44b64db484f24a46bf0b7ced46b519bf` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:58.857430 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c925488-8fe3-4b60-8a80-4f7c11473f99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:58.859219 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c925488-8fe3-4b60-8a80-4f7c11473f99 service neutron] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:58.864698 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b083fb-97cb-44a6-ab98-2473a6d64155 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ec81b012136413ba8fb9a9f61ca6625, user_id=534a012969ef4493868aee2737da0a74, role_id=44b64db484f24a46bf0b7ced46b519bf)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:58.867395 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b083fb-97cb-44a6-ab98-2473a6d64155 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:58.873097 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 971/1934] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:42:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 08:42:58.979721 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 964/1935] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 08:42:58 2022] PUT /identity/v3/projects/1ec81b012136413ba8fb9a9f61ca6625/users/534a012969ef4493868aee2737da0a74/roles/44b64db484f24a46bf0b7ced46b519bf => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 08:42:58.991230 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9665cb6-5f6f-4a5e-be02-4a87cc7ea023 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:58.991663 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9665cb6-5f6f-4a5e-be02-4a87cc7ea023 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:58.992016 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9665cb6-5f6f-4a5e-be02-4a87cc7ea023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:59.024117 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-d9665cb6-5f6f-4a5e-be02-4a87cc7ea023 None None] MFA Rules not processed for user `534a012969ef4493868aee2737da0a74`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:42:59.025939 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-d9665cb6-5f6f-4a5e-be02-4a87cc7ea023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:42:59.163509 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 972/1936] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:42:58 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:42:59.176034 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b083fb-97cb-44a6-ab98-2473a6d64155 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:59.179250 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-82b083fb-97cb-44a6-ab98-2473a6d64155 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:42:59.265976 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b083fb-97cb-44a6-ab98-2473a6d64155 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:59.270188 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e83b20-0f43-419d-b1a2-8940274e85a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ba74e28ced4cbba857f54222dfbe41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:59.270920 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e83b20-0f43-419d-b1a2-8940274e85a0 service cinder] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:59.271197 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e83b20-0f43-419d-b1a2-8940274e85a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:59.271464 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e83b20-0f43-419d-b1a2-8940274e85a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:59.284147 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e83b20-0f43-419d-b1a2-8940274e85a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:59.285784 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e83b20-0f43-419d-b1a2-8940274e85a0 service cinder] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:59.297040 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 965/1937] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:42:59 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:42:59.363382 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9665cb6-5f6f-4a5e-be02-4a87cc7ea023 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:59.365535 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-d9665cb6-5f6f-4a5e-be02-4a87cc7ea023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:42:59.583363 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9665cb6-5f6f-4a5e-be02-4a87cc7ea023 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:42:59.598675 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5360ce1-ccb4-4c3a-ad1d-5eb5a141672a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:42:59.599557 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5360ce1-ccb4-4c3a-ad1d-5eb5a141672a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:42:59.599916 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5360ce1-ccb4-4c3a-ad1d-5eb5a141672a admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:42:59.600196 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5360ce1-ccb4-4c3a-ad1d-5eb5a141672a admin admin] PATH_INFO: `/v3/projects/f20c187d7b6d411cba09bf374ccc5390` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:42:59.601418 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5360ce1-ccb4-4c3a-ad1d-5eb5a141672a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f20c187d7b6d411cba09bf374ccc5390)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:42:59.607670 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5360ce1-ccb4-4c3a-ad1d-5eb5a141672a admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:42:59.626403 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:42:59.793548 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-c5360ce1-ccb4-4c3a-ad1d-5eb5a141672a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f20c187d7b6d411cba09bf374ccc5390', 'request_id': 'req-c5360ce1-ccb4-4c3a-ad1d-5eb5a141672a'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:59.796697 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-c5360ce1-ccb4-4c3a-ad1d-5eb5a141672a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f20c187d7b6d411cba09bf374ccc5390', 'request_id': 'req-c5360ce1-ccb4-4c3a-ad1d-5eb5a141672a'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:59.875247 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-c5360ce1-ccb4-4c3a-ad1d-5eb5a141672a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f20c187d7b6d411cba09bf374ccc5390', 'request_id': 'req-c5360ce1-ccb4-4c3a-ad1d-5eb5a141672a'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:59.889885 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-c5360ce1-ccb4-4c3a-ad1d-5eb5a141672a admin admin] The token cache is being invalidate because project f20c187d7b6d411cba09bf374ccc5390 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 08:42:59.890370 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-c5360ce1-ccb4-4c3a-ad1d-5eb5a141672a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:59.891141 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-c5360ce1-ccb4-4c3a-ad1d-5eb5a141672a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:42:59.893847 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 973/1938] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 08:42:59 2022] DELETE /identity/v3/projects/f20c187d7b6d411cba09bf374ccc5390 => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:42:59.898897 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e83b20-0f43-419d-b1a2-8940274e85a0 service cinder] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:42:59.910075 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e83b20-0f43-419d-b1a2-8940274e85a0 service cinder] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:00.006113 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b485cc-4a71-4808-b50e-53ca2fb75044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:00.006891 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b485cc-4a71-4808-b50e-53ca2fb75044 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:00.007030 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b485cc-4a71-4808-b50e-53ca2fb75044 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:00.007480 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b485cc-4a71-4808-b50e-53ca2fb75044 admin admin] PATH_INFO: `/v3/users/246efda3b947486a8c054a0e6583080a` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:00.008686 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b485cc-4a71-4808-b50e-53ca2fb75044 admin admin] RBAC: Authorizing `identity:delete_user(user_id=246efda3b947486a8c054a0e6583080a)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:00.010824 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b485cc-4a71-4808-b50e-53ca2fb75044 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:00.169733 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-14b485cc-4a71-4808-b50e-53ca2fb75044 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '246efda3b947486a8c054a0e6583080a'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:00.181332 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-14b485cc-4a71-4808-b50e-53ca2fb75044 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '246efda3b947486a8c054a0e6583080a'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:00.187982 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-14b485cc-4a71-4808-b50e-53ca2fb75044 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '246efda3b947486a8c054a0e6583080a'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:00.188641 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-14b485cc-4a71-4808-b50e-53ca2fb75044 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '246efda3b947486a8c054a0e6583080a'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:00.189253 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-14b485cc-4a71-4808-b50e-53ca2fb75044 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '246efda3b947486a8c054a0e6583080a'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:00.192907 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:43:00.286173 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 966/1939] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 08:42:59 2022] DELETE /identity/v3/users/246efda3b947486a8c054a0e6583080a => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:43:00.932039 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5360ce1-ccb4-4c3a-ad1d-5eb5a141672a admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:00.934236 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-c5360ce1-ccb4-4c3a-ad1d-5eb5a141672a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:00.953057 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5360ce1-ccb4-4c3a-ad1d-5eb5a141672a admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:01.027549 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55473d86-7b73-418a-bb8e-e35dc8a73161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:01.028192 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55473d86-7b73-418a-bb8e-e35dc8a73161 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:01.028478 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55473d86-7b73-418a-bb8e-e35dc8a73161 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:01.028749 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55473d86-7b73-418a-bb8e-e35dc8a73161 admin admin] PATH_INFO: `/v3/projects/73aeab67d55e4bb284a1ca2e14aab780` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:01.030220 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55473d86-7b73-418a-bb8e-e35dc8a73161 admin admin] RBAC: Authorizing `identity:delete_project(project_id=73aeab67d55e4bb284a1ca2e14aab780)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:01.032176 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55473d86-7b73-418a-bb8e-e35dc8a73161 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:01.166015 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-55473d86-7b73-418a-bb8e-e35dc8a73161 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73aeab67d55e4bb284a1ca2e14aab780', 'request_id': 'req-55473d86-7b73-418a-bb8e-e35dc8a73161'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:01.172430 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-55473d86-7b73-418a-bb8e-e35dc8a73161 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73aeab67d55e4bb284a1ca2e14aab780', 'request_id': 'req-55473d86-7b73-418a-bb8e-e35dc8a73161'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:01.252342 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-55473d86-7b73-418a-bb8e-e35dc8a73161 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73aeab67d55e4bb284a1ca2e14aab780', 'request_id': 'req-55473d86-7b73-418a-bb8e-e35dc8a73161'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:01.265749 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-55473d86-7b73-418a-bb8e-e35dc8a73161 admin admin] The token cache is being invalidate because project 73aeab67d55e4bb284a1ca2e14aab780 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 08:43:01.266297 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-55473d86-7b73-418a-bb8e-e35dc8a73161 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:01.267114 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-55473d86-7b73-418a-bb8e-e35dc8a73161 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:01.268661 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 974/1940] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 08:43:00 2022] DELETE /identity/v3/projects/73aeab67d55e4bb284a1ca2e14aab780 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:43:01.273680 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46b07ed-5dd3-4db5-ab58-622ffd6c6d1e None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:01.274146 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46b07ed-5dd3-4db5-ab58-622ffd6c6d1e None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:01.274237 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46b07ed-5dd3-4db5-ab58-622ffd6c6d1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:01.312800 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-b46b07ed-5dd3-4db5-ab58-622ffd6c6d1e None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:43:01.313416 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-b46b07ed-5dd3-4db5-ab58-622ffd6c6d1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:01.403981 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 967/1941] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:43:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 08:43:01.412491 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26609e4c-05a5-4eee-b2bb-d6bc7a6ba9ab None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:01.412782 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26609e4c-05a5-4eee-b2bb-d6bc7a6ba9ab None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:01.413084 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26609e4c-05a5-4eee-b2bb-d6bc7a6ba9ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:01.434225 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-26609e4c-05a5-4eee-b2bb-d6bc7a6ba9ab None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:43:01.435525 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-26609e4c-05a5-4eee-b2bb-d6bc7a6ba9ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:01.445160 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 975/1942] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:43:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 08:43:01.449252 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46b07ed-5dd3-4db5-ab58-622ffd6c6d1e None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:01.458684 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46b07ed-5dd3-4db5-ab58-622ffd6c6d1e None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:01.463420 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2572d4-fffe-4d77-be76-b7dd7655fa6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:01.464017 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2572d4-fffe-4d77-be76-b7dd7655fa6a admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:01.464294 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2572d4-fffe-4d77-be76-b7dd7655fa6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:01.464560 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2572d4-fffe-4d77-be76-b7dd7655fa6a admin admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:01.465262 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2572d4-fffe-4d77-be76-b7dd7655fa6a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:43:01.465561 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2572d4-fffe-4d77-be76-b7dd7655fa6a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:01.467176 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2572d4-fffe-4d77-be76-b7dd7655fa6a admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:01.479136 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 968/1943] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Aug 10 08:43:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:43:01.483427 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26609e4c-05a5-4eee-b2bb-d6bc7a6ba9ab None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:01.493214 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26609e4c-05a5-4eee-b2bb-d6bc7a6ba9ab None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:01.498086 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e49c8ec-4a10-4b90-983f-578aa08c5a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:01.498682 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e49c8ec-4a10-4b90-983f-578aa08c5a29 admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:01.498960 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e49c8ec-4a10-4b90-983f-578aa08c5a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:01.499231 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e49c8ec-4a10-4b90-983f-578aa08c5a29 admin admin] PATH_INFO: `/v3/projects` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:01.499902 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e49c8ec-4a10-4b90-983f-578aa08c5a29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:01.501951 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e49c8ec-4a10-4b90-983f-578aa08c5a29 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:01.518186 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:43:01.627724 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 976/1944] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 08:43:01 2022] POST /identity/v3/projects => generated 385 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:43:01.632683 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2572d4-fffe-4d77-be76-b7dd7655fa6a admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:01.647305 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2572d4-fffe-4d77-be76-b7dd7655fa6a admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:01.715683 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afb0ad4-4b09-4b7b-836e-bb8c4cf42747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:01.716288 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afb0ad4-4b09-4b7b-836e-bb8c4cf42747 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:01.716565 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afb0ad4-4b09-4b7b-836e-bb8c4cf42747 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:01.716832 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afb0ad4-4b09-4b7b-836e-bb8c4cf42747 admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:01.717488 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afb0ad4-4b09-4b7b-836e-bb8c4cf42747 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:01.719491 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afb0ad4-4b09-4b7b-836e-bb8c4cf42747 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:01.829078 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 969/1945] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 08:43:01 2022] POST /identity/v3/users => generated 293 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 08:43:01.830274 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e49c8ec-4a10-4b90-983f-578aa08c5a29 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:01.841529 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e49c8ec-4a10-4b90-983f-578aa08c5a29 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:01.846274 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f34b568-9692-4a04-bb61-57fa9a9d4518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:01.847021 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f34b568-9692-4a04-bb61-57fa9a9d4518 admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:01.847305 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f34b568-9692-4a04-bb61-57fa9a9d4518 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:01.847579 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f34b568-9692-4a04-bb61-57fa9a9d4518 admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:01.848769 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f34b568-9692-4a04-bb61-57fa9a9d4518 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:01.850448 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f34b568-9692-4a04-bb61-57fa9a9d4518 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:01.862321 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 977/1946] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:43:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:43:01.866787 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afb0ad4-4b09-4b7b-836e-bb8c4cf42747 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:01.878063 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afb0ad4-4b09-4b7b-836e-bb8c4cf42747 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:01.883024 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53672b8-4f53-474f-8d7a-e76e643774bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:01.884632 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53672b8-4f53-474f-8d7a-e76e643774bf admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:01.884632 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53672b8-4f53-474f-8d7a-e76e643774bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:01.884632 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53672b8-4f53-474f-8d7a-e76e643774bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:01.885032 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53672b8-4f53-474f-8d7a-e76e643774bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:01.887130 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53672b8-4f53-474f-8d7a-e76e643774bf admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:01.903905 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 970/1947] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:43:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:43:01.913205 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f34b568-9692-4a04-bb61-57fa9a9d4518 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:01.921844 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f34b568-9692-4a04-bb61-57fa9a9d4518 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:01.927163 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3076989d-8353-4170-9c37-5cb537f94a94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:01.928679 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3076989d-8353-4170-9c37-5cb537f94a94 admin admin] REQUEST_METHOD: `PUT` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:01.928679 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3076989d-8353-4170-9c37-5cb537f94a94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:01.928679 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3076989d-8353-4170-9c37-5cb537f94a94 admin admin] PATH_INFO: `/v3/projects/de75c7d03fca49e2aa8112a63af4257b/users/cecef61ff22e4444986975432c86d7e1/roles/44b64db484f24a46bf0b7ced46b519bf` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:01.945165 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3076989d-8353-4170-9c37-5cb537f94a94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de75c7d03fca49e2aa8112a63af4257b, user_id=cecef61ff22e4444986975432c86d7e1, role_id=44b64db484f24a46bf0b7ced46b519bf)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:01.947920 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3076989d-8353-4170-9c37-5cb537f94a94 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:02.069933 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 978/1948] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 08:43:01 2022] PUT /identity/v3/projects/de75c7d03fca49e2aa8112a63af4257b/users/cecef61ff22e4444986975432c86d7e1/roles/44b64db484f24a46bf0b7ced46b519bf => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:43:02.083616 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8335ea7-89bd-4bf1-9e8a-6db989891b85 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:02.083985 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8335ea7-89bd-4bf1-9e8a-6db989891b85 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:02.084328 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8335ea7-89bd-4bf1-9e8a-6db989891b85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:02.121810 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-f8335ea7-89bd-4bf1-9e8a-6db989891b85 None None] MFA Rules not processed for user `cecef61ff22e4444986975432c86d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:43:02.123279 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-f8335ea7-89bd-4bf1-9e8a-6db989891b85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:02.182712 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:43:02.204866 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 971/1949] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:43:02 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:43:02.210123 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3076989d-8353-4170-9c37-5cb537f94a94 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:02.293131 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3076989d-8353-4170-9c37-5cb537f94a94 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:02.321432 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a56b9a-656c-44cc-b66a-f2997ae12c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:02.322066 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a56b9a-656c-44cc-b66a-f2997ae12c2b admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:02.322390 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a56b9a-656c-44cc-b66a-f2997ae12c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:02.322660 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a56b9a-656c-44cc-b66a-f2997ae12c2b admin admin] PATH_INFO: `/v3/projects` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:02.323299 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a56b9a-656c-44cc-b66a-f2997ae12c2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:02.326518 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a56b9a-656c-44cc-b66a-f2997ae12c2b admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:02.437408 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 979/1950] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 08:43:02 2022] POST /identity/v3/projects => generated 387 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:43:02.438797 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8335ea7-89bd-4bf1-9e8a-6db989891b85 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:02.455489 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8335ea7-89bd-4bf1-9e8a-6db989891b85 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:02.558799 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61764bf3-e5bd-4ec8-8ddc-38867a4ee5c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:02.559709 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61764bf3-e5bd-4ec8-8ddc-38867a4ee5c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:02.560137 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61764bf3-e5bd-4ec8-8ddc-38867a4ee5c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:02.560553 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61764bf3-e5bd-4ec8-8ddc-38867a4ee5c8 admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:02.561362 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61764bf3-e5bd-4ec8-8ddc-38867a4ee5c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:02.563499 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61764bf3-e5bd-4ec8-8ddc-38867a4ee5c8 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:02.691281 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a56b9a-656c-44cc-b66a-f2997ae12c2b admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:02.701544 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 972/1951] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 08:43:02 2022] POST /identity/v3/users => generated 300 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:43:02.706962 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a56b9a-656c-44cc-b66a-f2997ae12c2b admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:02.714864 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032e3bcb-7b65-457f-bab1-9882052a4ef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:02.716241 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032e3bcb-7b65-457f-bab1-9882052a4ef5 admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:02.716623 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032e3bcb-7b65-457f-bab1-9882052a4ef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:02.717123 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032e3bcb-7b65-457f-bab1-9882052a4ef5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:02.724976 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032e3bcb-7b65-457f-bab1-9882052a4ef5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:02.724976 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032e3bcb-7b65-457f-bab1-9882052a4ef5 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:02.741771 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61764bf3-e5bd-4ec8-8ddc-38867a4ee5c8 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:02.752653 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 980/1952] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:43:02 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:43:02.757058 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61764bf3-e5bd-4ec8-8ddc-38867a4ee5c8 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:02.762545 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94974ce5-d9cc-49bd-baa4-9b906dbf94bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:02.763295 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94974ce5-d9cc-49bd-baa4-9b906dbf94bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:02.763677 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94974ce5-d9cc-49bd-baa4-9b906dbf94bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:02.764020 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94974ce5-d9cc-49bd-baa4-9b906dbf94bb admin admin] PATH_INFO: `/v3/projects/44d4f1032ea54db2b51b6280cfe13c0b/users/2c776badddd844c6b80bae8022054524/roles/ec3751692d9c4ecf81b4a71c936c46be` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:02.790040 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94974ce5-d9cc-49bd-baa4-9b906dbf94bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=44d4f1032ea54db2b51b6280cfe13c0b, user_id=2c776badddd844c6b80bae8022054524, role_id=ec3751692d9c4ecf81b4a71c936c46be)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:02.810315 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94974ce5-d9cc-49bd-baa4-9b906dbf94bb admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:02.881945 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 973/1953] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 08:43:02 2022] PUT /identity/v3/projects/44d4f1032ea54db2b51b6280cfe13c0b/users/2c776badddd844c6b80bae8022054524/roles/ec3751692d9c4ecf81b4a71c936c46be => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:43:02.887231 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032e3bcb-7b65-457f-bab1-9882052a4ef5 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:02.893669 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94974ce5-d9cc-49bd-baa4-9b906dbf94bb admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:02.897761 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-94974ce5-d9cc-49bd-baa4-9b906dbf94bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:02.904609 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032e3bcb-7b65-457f-bab1-9882052a4ef5 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:03.048188 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94974ce5-d9cc-49bd-baa4-9b906dbf94bb admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:03.049512 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8103c51-c0b6-40d3-8698-9cae637022d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:03.050676 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8103c51-c0b6-40d3-8698-9cae637022d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:03.051679 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8103c51-c0b6-40d3-8698-9cae637022d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:03.052166 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8103c51-c0b6-40d3-8698-9cae637022d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:03.053038 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8103c51-c0b6-40d3-8698-9cae637022d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:03.054843 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8103c51-c0b6-40d3-8698-9cae637022d6 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:03.065899 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1c97b8-5b55-4a20-a423-91251ede8714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:03.068129 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1c97b8-5b55-4a20-a423-91251ede8714 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:03.068609 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1c97b8-5b55-4a20-a423-91251ede8714 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:03.069199 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1c97b8-5b55-4a20-a423-91251ede8714 admin admin] PATH_INFO: `/v3/users/0ae16edf81394419895607ce058add00` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:03.071992 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1c97b8-5b55-4a20-a423-91251ede8714 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ae16edf81394419895607ce058add00)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:03.074187 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1c97b8-5b55-4a20-a423-91251ede8714 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:03.083282 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 981/1954] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:43:02 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:43:03.084441 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8103c51-c0b6-40d3-8698-9cae637022d6 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:03.107451 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8103c51-c0b6-40d3-8698-9cae637022d6 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:03.121149 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cff203-d3fd-4764-9fd7-f59a632bb216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:03.122462 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cff203-d3fd-4764-9fd7-f59a632bb216 admin admin] REQUEST_METHOD: `PUT` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:03.122927 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cff203-d3fd-4764-9fd7-f59a632bb216 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:03.123379 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cff203-d3fd-4764-9fd7-f59a632bb216 admin admin] PATH_INFO: `/v3/projects/44d4f1032ea54db2b51b6280cfe13c0b/users/2c776badddd844c6b80bae8022054524/roles/ec3751692d9c4ecf81b4a71c936c46be` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:03.126189 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cff203-d3fd-4764-9fd7-f59a632bb216 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44d4f1032ea54db2b51b6280cfe13c0b, user_id=2c776badddd844c6b80bae8022054524, role_id=ec3751692d9c4ecf81b4a71c936c46be)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:03.130152 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cff203-d3fd-4764-9fd7-f59a632bb216 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:03.143013 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 982/1955] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 08:43:03 2022] PUT /identity/v3/projects/44d4f1032ea54db2b51b6280cfe13c0b/users/2c776badddd844c6b80bae8022054524/roles/ec3751692d9c4ecf81b4a71c936c46be => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:43:03.154626 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa404b-aec3-4d4d-94da-3da1c5a8e496 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:03.154913 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa404b-aec3-4d4d-94da-3da1c5a8e496 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:03.155183 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa404b-aec3-4d4d-94da-3da1c5a8e496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:03.185729 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-58aa404b-aec3-4d4d-94da-3da1c5a8e496 None None] MFA Rules not processed for user `2c776badddd844c6b80bae8022054524`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:43:03.187134 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-58aa404b-aec3-4d4d-94da-3da1c5a8e496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:03.192873 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:43:03.256537 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-5f1c97b8-5b55-4a20-a423-91251ede8714 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ae16edf81394419895607ce058add00'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:03.271034 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:43:03.277332 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-5f1c97b8-5b55-4a20-a423-91251ede8714 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ae16edf81394419895607ce058add00'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:03.288781 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-5f1c97b8-5b55-4a20-a423-91251ede8714 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ae16edf81394419895607ce058add00'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:03.290509 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-5f1c97b8-5b55-4a20-a423-91251ede8714 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ae16edf81394419895607ce058add00'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:03.291518 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-5f1c97b8-5b55-4a20-a423-91251ede8714 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ae16edf81394419895607ce058add00'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:03.294200 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 983/1956] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:43:03 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:43:03.308402 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aa404b-aec3-4d4d-94da-3da1c5a8e496 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:03.312374 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-58aa404b-aec3-4d4d-94da-3da1c5a8e496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:03.381296 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 974/1957] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 08:43:02 2022] DELETE /identity/v3/users/0ae16edf81394419895607ce058add00 => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:43:03.388667 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd1cde2-2f15-4e44-af93-3483e06ac544 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:03.388667 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd1cde2-2f15-4e44-af93-3483e06ac544 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:03.388667 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd1cde2-2f15-4e44-af93-3483e06ac544 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:03.407487 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aa404b-aec3-4d4d-94da-3da1c5a8e496 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:03.416997 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-4cd1cde2-2f15-4e44-af93-3483e06ac544 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:43:03.418273 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-4cd1cde2-2f15-4e44-af93-3483e06ac544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:03.472140 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb57774-03d4-4391-af71-805a0c222361 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ba74e28ced4cbba857f54222dfbe41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:03.473938 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb57774-03d4-4391-af71-805a0c222361 service cinder] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:03.474205 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb57774-03d4-4391-af71-805a0c222361 service cinder] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:03.474624 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb57774-03d4-4391-af71-805a0c222361 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:03.476621 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-2fb57774-03d4-4391-af71-805a0c222361 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:03.508940 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 975/1958] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:43:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:43:03.546256 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd1cde2-2f15-4e44-af93-3483e06ac544 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:03.548272 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-4cd1cde2-2f15-4e44-af93-3483e06ac544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:03.554682 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb57774-03d4-4391-af71-805a0c222361 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:03.556372 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb57774-03d4-4391-af71-805a0c222361 service cinder] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:03.570016 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 984/1959] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:43:03 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:43:03.603332 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb57774-03d4-4391-af71-805a0c222361 service cinder] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:03.605484 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-2fb57774-03d4-4391-af71-805a0c222361 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:03.624834 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb57774-03d4-4391-af71-805a0c222361 service cinder] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:03.631974 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53e5f52-a876-4fbb-beee-0d42c6f989f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:03.632900 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53e5f52-a876-4fbb-beee-0d42c6f989f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:03.633217 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53e5f52-a876-4fbb-beee-0d42c6f989f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:03.633987 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53e5f52-a876-4fbb-beee-0d42c6f989f4 admin admin] PATH_INFO: `/v3/users/cecef61ff22e4444986975432c86d7e1` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:03.635204 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53e5f52-a876-4fbb-beee-0d42c6f989f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cecef61ff22e4444986975432c86d7e1)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:03.637316 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53e5f52-a876-4fbb-beee-0d42c6f989f4 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:03.642502 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:43:03.652542 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd1cde2-2f15-4e44-af93-3483e06ac544 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:03.657277 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d694beb-4515-427d-a937-04fda0c51c98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:03.658222 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d694beb-4515-427d-a937-04fda0c51c98 service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:03.658222 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d694beb-4515-427d-a937-04fda0c51c98 service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:03.658524 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d694beb-4515-427d-a937-04fda0c51c98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:03.671615 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d694beb-4515-427d-a937-04fda0c51c98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:03.673278 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d694beb-4515-427d-a937-04fda0c51c98 service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:03.684787 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 976/1960] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:43:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:43:03.855374 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-c53e5f52-a876-4fbb-beee-0d42c6f989f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cecef61ff22e4444986975432c86d7e1'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:03.877816 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-c53e5f52-a876-4fbb-beee-0d42c6f989f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cecef61ff22e4444986975432c86d7e1'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:03.886860 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-c53e5f52-a876-4fbb-beee-0d42c6f989f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cecef61ff22e4444986975432c86d7e1'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:03.974123 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-c53e5f52-a876-4fbb-beee-0d42c6f989f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cecef61ff22e4444986975432c86d7e1'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:03.974824 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-c53e5f52-a876-4fbb-beee-0d42c6f989f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cecef61ff22e4444986975432c86d7e1'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:03.982220 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 985/1961] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 08:43:03 2022] DELETE /identity/v3/users/cecef61ff22e4444986975432c86d7e1 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:43:03.986730 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b86257-6eaa-4db8-87aa-c19ad28db9b2 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:03.987019 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b86257-6eaa-4db8-87aa-c19ad28db9b2 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:03.987305 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b86257-6eaa-4db8-87aa-c19ad28db9b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:04.012740 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-48b86257-6eaa-4db8-87aa-c19ad28db9b2 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:43:04.013994 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-48b86257-6eaa-4db8-87aa-c19ad28db9b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:04.088269 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 977/1962] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:43:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:43:04.097579 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53e5f52-a876-4fbb-beee-0d42c6f989f4 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:04.100375 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-c53e5f52-a876-4fbb-beee-0d42c6f989f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:04.200093 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53e5f52-a876-4fbb-beee-0d42c6f989f4 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:04.217900 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c35ff4-b234-49bf-833f-0285f98897c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:04.220064 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c35ff4-b234-49bf-833f-0285f98897c7 service neutron] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:04.223236 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c35ff4-b234-49bf-833f-0285f98897c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:04.223236 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c35ff4-b234-49bf-833f-0285f98897c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:04.254491 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c35ff4-b234-49bf-833f-0285f98897c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:04.256117 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c35ff4-b234-49bf-833f-0285f98897c7 service neutron] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:04.283363 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 986/1963] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:43:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:43:04.507725 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b86257-6eaa-4db8-87aa-c19ad28db9b2 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:04.509381 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-48b86257-6eaa-4db8-87aa-c19ad28db9b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:04.532080 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b86257-6eaa-4db8-87aa-c19ad28db9b2 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:04.540483 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07fc3a2-d6b6-49e1-84c6-59c6e9b6aac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:04.541240 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07fc3a2-d6b6-49e1-84c6-59c6e9b6aac6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:04.541575 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07fc3a2-d6b6-49e1-84c6-59c6e9b6aac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:04.541906 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07fc3a2-d6b6-49e1-84c6-59c6e9b6aac6 admin admin] PATH_INFO: `/v3/projects/867b600e79d040ccb23dfd44cd36c0cc` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:04.543671 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07fc3a2-d6b6-49e1-84c6-59c6e9b6aac6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=867b600e79d040ccb23dfd44cd36c0cc)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:04.545864 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07fc3a2-d6b6-49e1-84c6-59c6e9b6aac6 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:04.565737 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:43:04.643948 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-b07fc3a2-d6b6-49e1-84c6-59c6e9b6aac6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '867b600e79d040ccb23dfd44cd36c0cc', 'request_id': 'req-b07fc3a2-d6b6-49e1-84c6-59c6e9b6aac6'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:04.698319 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-b07fc3a2-d6b6-49e1-84c6-59c6e9b6aac6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '867b600e79d040ccb23dfd44cd36c0cc', 'request_id': 'req-b07fc3a2-d6b6-49e1-84c6-59c6e9b6aac6'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:04.706325 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-b07fc3a2-d6b6-49e1-84c6-59c6e9b6aac6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '867b600e79d040ccb23dfd44cd36c0cc', 'request_id': 'req-b07fc3a2-d6b6-49e1-84c6-59c6e9b6aac6'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:04.726788 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-b07fc3a2-d6b6-49e1-84c6-59c6e9b6aac6 admin admin] The token cache is being invalidate because project 867b600e79d040ccb23dfd44cd36c0cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 08:43:04.727144 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-b07fc3a2-d6b6-49e1-84c6-59c6e9b6aac6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:04.731628 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-b07fc3a2-d6b6-49e1-84c6-59c6e9b6aac6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:04.733654 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 978/1964] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 08:43:04 2022] DELETE /identity/v3/projects/867b600e79d040ccb23dfd44cd36c0cc => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:43:04.737311 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c35ff4-b234-49bf-833f-0285f98897c7 service neutron] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:04.760507 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c35ff4-b234-49bf-833f-0285f98897c7 service neutron] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:04.794324 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07fc3a2-d6b6-49e1-84c6-59c6e9b6aac6 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:04.796185 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-b07fc3a2-d6b6-49e1-84c6-59c6e9b6aac6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:04.874403 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7da0fdf-4a38-47b6-bfc1-f296987a8b6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:04.875062 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7da0fdf-4a38-47b6-bfc1-f296987a8b6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:04.875824 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7da0fdf-4a38-47b6-bfc1-f296987a8b6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:04.876006 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7da0fdf-4a38-47b6-bfc1-f296987a8b6e admin admin] PATH_INFO: `/v3/users/ad6f1cbb718f450ca0809917bf2af7cf` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:04.880676 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7da0fdf-4a38-47b6-bfc1-f296987a8b6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad6f1cbb718f450ca0809917bf2af7cf)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:04.883887 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7da0fdf-4a38-47b6-bfc1-f296987a8b6e admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:04.916905 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07fc3a2-d6b6-49e1-84c6-59c6e9b6aac6 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:04.922173 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d019db-dc0e-427d-b982-76989f46e694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:04.922958 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d019db-dc0e-427d-b982-76989f46e694 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:04.923302 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d019db-dc0e-427d-b982-76989f46e694 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:04.923730 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d019db-dc0e-427d-b982-76989f46e694 admin admin] PATH_INFO: `/v3/projects/de75c7d03fca49e2aa8112a63af4257b` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:04.930240 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d019db-dc0e-427d-b982-76989f46e694 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de75c7d03fca49e2aa8112a63af4257b)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:04.934979 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d019db-dc0e-427d-b982-76989f46e694 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:04.973579 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:43:05.048865 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-b7da0fdf-4a38-47b6-bfc1-f296987a8b6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad6f1cbb718f450ca0809917bf2af7cf'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:05.067991 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-b7da0fdf-4a38-47b6-bfc1-f296987a8b6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad6f1cbb718f450ca0809917bf2af7cf'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:05.075881 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-b7da0fdf-4a38-47b6-bfc1-f296987a8b6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad6f1cbb718f450ca0809917bf2af7cf'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:05.091044 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-81d019db-dc0e-427d-b982-76989f46e694 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de75c7d03fca49e2aa8112a63af4257b', 'request_id': 'req-81d019db-dc0e-427d-b982-76989f46e694'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:05.155040 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-b7da0fdf-4a38-47b6-bfc1-f296987a8b6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad6f1cbb718f450ca0809917bf2af7cf'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:05.156576 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-b7da0fdf-4a38-47b6-bfc1-f296987a8b6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad6f1cbb718f450ca0809917bf2af7cf'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:05.168650 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 987/1965] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 08:43:04 2022] DELETE /identity/v3/users/ad6f1cbb718f450ca0809917bf2af7cf => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:43:05.217992 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-81d019db-dc0e-427d-b982-76989f46e694 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de75c7d03fca49e2aa8112a63af4257b', 'request_id': 'req-81d019db-dc0e-427d-b982-76989f46e694'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:05.229527 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-81d019db-dc0e-427d-b982-76989f46e694 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de75c7d03fca49e2aa8112a63af4257b', 'request_id': 'req-81d019db-dc0e-427d-b982-76989f46e694'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:05.260415 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-81d019db-dc0e-427d-b982-76989f46e694 admin admin] The token cache is being invalidate because project de75c7d03fca49e2aa8112a63af4257b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 08:43:05.260415 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-81d019db-dc0e-427d-b982-76989f46e694 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:05.261042 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-81d019db-dc0e-427d-b982-76989f46e694 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:05.268668 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 979/1966] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 08:43:04 2022] DELETE /identity/v3/projects/de75c7d03fca49e2aa8112a63af4257b => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 08:43:05.270628 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7da0fdf-4a38-47b6-bfc1-f296987a8b6e admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:05.272517 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-b7da0fdf-4a38-47b6-bfc1-f296987a8b6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:05.374024 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7da0fdf-4a38-47b6-bfc1-f296987a8b6e admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:05.381649 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d881595-dc42-4ea3-af45-25aff0c9fda4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:05.382384 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d881595-dc42-4ea3-af45-25aff0c9fda4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:05.382923 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d881595-dc42-4ea3-af45-25aff0c9fda4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:05.383311 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d881595-dc42-4ea3-af45-25aff0c9fda4 admin admin] PATH_INFO: `/v3/users/2c776badddd844c6b80bae8022054524` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:05.384610 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d881595-dc42-4ea3-af45-25aff0c9fda4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c776badddd844c6b80bae8022054524)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:05.387066 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d881595-dc42-4ea3-af45-25aff0c9fda4 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:05.635708 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-1d881595-dc42-4ea3-af45-25aff0c9fda4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c776badddd844c6b80bae8022054524'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:05.651377 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-1d881595-dc42-4ea3-af45-25aff0c9fda4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c776badddd844c6b80bae8022054524'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:05.658935 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-1d881595-dc42-4ea3-af45-25aff0c9fda4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c776badddd844c6b80bae8022054524'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:05.738096 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-1d881595-dc42-4ea3-af45-25aff0c9fda4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c776badddd844c6b80bae8022054524'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:05.740097 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-1d881595-dc42-4ea3-af45-25aff0c9fda4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c776badddd844c6b80bae8022054524'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:05.747464 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d019db-dc0e-427d-b982-76989f46e694 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:05.749115 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-81d019db-dc0e-427d-b982-76989f46e694 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:05.749746 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 988/1967] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 08:43:05 2022] DELETE /identity/v3/users/2c776badddd844c6b80bae8022054524 => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:43:05.824240 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:43:05.871875 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d019db-dc0e-427d-b982-76989f46e694 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:05.940810 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688b44c5-58c9-4108-8f74-ab5b0a6ce117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:05.941999 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b44c5-58c9-4108-8f74-ab5b0a6ce117 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:05.942604 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b44c5-58c9-4108-8f74-ab5b0a6ce117 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:05.942988 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b44c5-58c9-4108-8f74-ab5b0a6ce117 admin admin] PATH_INFO: `/v3/projects/7c90d44540f0458e95a2e5c3c2f25840` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:05.944477 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688b44c5-58c9-4108-8f74-ab5b0a6ce117 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c90d44540f0458e95a2e5c3c2f25840)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:05.947055 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688b44c5-58c9-4108-8f74-ab5b0a6ce117 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:06.175938 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-688b44c5-58c9-4108-8f74-ab5b0a6ce117 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c90d44540f0458e95a2e5c3c2f25840', 'request_id': 'req-688b44c5-58c9-4108-8f74-ab5b0a6ce117'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:06.228819 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d881595-dc42-4ea3-af45-25aff0c9fda4 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:06.236105 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-1d881595-dc42-4ea3-af45-25aff0c9fda4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:06.245669 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-688b44c5-58c9-4108-8f74-ab5b0a6ce117 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c90d44540f0458e95a2e5c3c2f25840', 'request_id': 'req-688b44c5-58c9-4108-8f74-ab5b0a6ce117'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:06.251370 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d881595-dc42-4ea3-af45-25aff0c9fda4 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:06.256437 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-688b44c5-58c9-4108-8f74-ab5b0a6ce117 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c90d44540f0458e95a2e5c3c2f25840', 'request_id': 'req-688b44c5-58c9-4108-8f74-ab5b0a6ce117'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:06.275750 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-688b44c5-58c9-4108-8f74-ab5b0a6ce117 admin admin] The token cache is being invalidate because project 7c90d44540f0458e95a2e5c3c2f25840 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 08:43:06.275891 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:43:06.276070 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-688b44c5-58c9-4108-8f74-ab5b0a6ce117 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:06.276809 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-688b44c5-58c9-4108-8f74-ab5b0a6ce117 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:06.278371 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 980/1968] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 08:43:05 2022] DELETE /identity/v3/projects/7c90d44540f0458e95a2e5c3c2f25840 => generated 0 bytes in 532 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 08:43:06.282463 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688b44c5-58c9-4108-8f74-ab5b0a6ce117 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:06.284207 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-688b44c5-58c9-4108-8f74-ab5b0a6ce117 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:06.321167 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fb5483-229d-409a-9d2a-2455a5d8e896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:06.322045 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fb5483-229d-409a-9d2a-2455a5d8e896 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:06.322387 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fb5483-229d-409a-9d2a-2455a5d8e896 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:06.322788 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fb5483-229d-409a-9d2a-2455a5d8e896 admin admin] PATH_INFO: `/v3/projects/44d4f1032ea54db2b51b6280cfe13c0b` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:06.324061 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fb5483-229d-409a-9d2a-2455a5d8e896 admin admin] RBAC: Authorizing `identity:delete_project(project_id=44d4f1032ea54db2b51b6280cfe13c0b)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:06.328129 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fb5483-229d-409a-9d2a-2455a5d8e896 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:06.363019 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688b44c5-58c9-4108-8f74-ab5b0a6ce117 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:06.368985 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7570e1f9-3c8d-4ec6-92f7-7240de8d1463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:06.369656 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7570e1f9-3c8d-4ec6-92f7-7240de8d1463 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:06.369936 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7570e1f9-3c8d-4ec6-92f7-7240de8d1463 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:06.370249 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7570e1f9-3c8d-4ec6-92f7-7240de8d1463 admin admin] PATH_INFO: `/v3/users/32e2909bf5814108a1c20b4717477654` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:06.371469 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7570e1f9-3c8d-4ec6-92f7-7240de8d1463 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32e2909bf5814108a1c20b4717477654)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:06.373741 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7570e1f9-3c8d-4ec6-92f7-7240de8d1463 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:06.388153 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:43:06.457282 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-97fb5483-229d-409a-9d2a-2455a5d8e896 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44d4f1032ea54db2b51b6280cfe13c0b', 'request_id': 'req-97fb5483-229d-409a-9d2a-2455a5d8e896'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:06.462866 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-97fb5483-229d-409a-9d2a-2455a5d8e896 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44d4f1032ea54db2b51b6280cfe13c0b', 'request_id': 'req-97fb5483-229d-409a-9d2a-2455a5d8e896'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:06.521559 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-97fb5483-229d-409a-9d2a-2455a5d8e896 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44d4f1032ea54db2b51b6280cfe13c0b', 'request_id': 'req-97fb5483-229d-409a-9d2a-2455a5d8e896'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:06.533580 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-97fb5483-229d-409a-9d2a-2455a5d8e896 admin admin] The token cache is being invalidate because project 44d4f1032ea54db2b51b6280cfe13c0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 08:43:06.533946 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-97fb5483-229d-409a-9d2a-2455a5d8e896 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:06.534653 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-97fb5483-229d-409a-9d2a-2455a5d8e896 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:06.537064 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 989/1969] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 08:43:06 2022] DELETE /identity/v3/projects/44d4f1032ea54db2b51b6280cfe13c0b => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:43:06.541505 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f92402-60db-460d-a0aa-70e4a550bdea None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:06.541793 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f92402-60db-460d-a0aa-70e4a550bdea None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:06.542070 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f92402-60db-460d-a0aa-70e4a550bdea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:06.568143 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-48f92402-60db-460d-a0aa-70e4a550bdea None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:43:06.569530 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-48f92402-60db-460d-a0aa-70e4a550bdea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:06.605245 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-7570e1f9-3c8d-4ec6-92f7-7240de8d1463 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32e2909bf5814108a1c20b4717477654'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:06.620498 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-7570e1f9-3c8d-4ec6-92f7-7240de8d1463 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32e2909bf5814108a1c20b4717477654'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:06.629879 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-7570e1f9-3c8d-4ec6-92f7-7240de8d1463 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32e2909bf5814108a1c20b4717477654'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:06.630681 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-7570e1f9-3c8d-4ec6-92f7-7240de8d1463 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32e2909bf5814108a1c20b4717477654'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:06.631377 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-7570e1f9-3c8d-4ec6-92f7-7240de8d1463 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32e2909bf5814108a1c20b4717477654'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:06.637753 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 990/1970] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:43:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 08:43:06.648714 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cc6820-e149-42b4-9f70-488433998f12 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:06.649034 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cc6820-e149-42b4-9f70-488433998f12 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:06.651618 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cc6820-e149-42b4-9f70-488433998f12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:06.669514 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-f3cc6820-e149-42b4-9f70-488433998f12 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:43:06.670713 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-f3cc6820-e149-42b4-9f70-488433998f12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:06.679141 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 991/1971] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:43:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:43:06.684375 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cc6820-e149-42b4-9f70-488433998f12 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:06.696573 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cc6820-e149-42b4-9f70-488433998f12 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:06.701346 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c455a2-564e-41a3-b94c-ddab42bde796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:06.701948 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c455a2-564e-41a3-b94c-ddab42bde796 admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:06.702227 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c455a2-564e-41a3-b94c-ddab42bde796 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:06.702510 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c455a2-564e-41a3-b94c-ddab42bde796 admin admin] PATH_INFO: `/v3/domains` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:06.703143 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c455a2-564e-41a3-b94c-ddab42bde796 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:43:06.703443 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c455a2-564e-41a3-b94c-ddab42bde796 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:06.705061 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c455a2-564e-41a3-b94c-ddab42bde796 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:06.718334 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 992/1972] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Aug 10 08:43:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:43:06.721093 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c455a2-564e-41a3-b94c-ddab42bde796 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:06.731680 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c455a2-564e-41a3-b94c-ddab42bde796 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:06.738018 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 981/1973] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 08:43:06 2022] DELETE /identity/v3/users/32e2909bf5814108a1c20b4717477654 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 08:43:06.763139 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:43:06.791232 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30023963-526d-4533-a00b-1967375a0c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:06.791840 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30023963-526d-4533-a00b-1967375a0c0f admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:06.795426 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30023963-526d-4533-a00b-1967375a0c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:06.795426 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30023963-526d-4533-a00b-1967375a0c0f admin admin] PATH_INFO: `/v3/projects` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:06.795426 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30023963-526d-4533-a00b-1967375a0c0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:06.795426 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30023963-526d-4533-a00b-1967375a0c0f admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:06.883743 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 993/1974] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 08:43:06 2022] POST /identity/v3/projects => generated 367 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 08:43:06.887910 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7570e1f9-3c8d-4ec6-92f7-7240de8d1463 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:06.898537 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7570e1f9-3c8d-4ec6-92f7-7240de8d1463 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:06.966685 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b8e610-9585-4e53-bf72-b4dc5424befd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:06.967320 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b8e610-9585-4e53-bf72-b4dc5424befd admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:06.967605 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b8e610-9585-4e53-bf72-b4dc5424befd admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:06.967879 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b8e610-9585-4e53-bf72-b4dc5424befd admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:06.968513 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b8e610-9585-4e53-bf72-b4dc5424befd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:06.970568 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b8e610-9585-4e53-bf72-b4dc5424befd admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:07.070834 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 982/1975] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 08:43:06 2022] POST /identity/v3/users => generated 284 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 08:43:07.074761 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30023963-526d-4533-a00b-1967375a0c0f admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:07.085379 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30023963-526d-4533-a00b-1967375a0c0f admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:07.089901 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69cfb1c-7f1e-4484-9f04-d312cb20ed97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:07.090570 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69cfb1c-7f1e-4484-9f04-d312cb20ed97 admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:07.090781 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69cfb1c-7f1e-4484-9f04-d312cb20ed97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:07.091050 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69cfb1c-7f1e-4484-9f04-d312cb20ed97 admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:07.091692 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69cfb1c-7f1e-4484-9f04-d312cb20ed97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:07.093323 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69cfb1c-7f1e-4484-9f04-d312cb20ed97 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:07.104292 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 994/1976] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:43:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:43:07.108945 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b8e610-9585-4e53-bf72-b4dc5424befd admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:07.120050 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b8e610-9585-4e53-bf72-b4dc5424befd admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:07.125328 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5131951c-29b7-4a8c-b221-055451be6bab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:07.125925 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5131951c-29b7-4a8c-b221-055451be6bab admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:07.126203 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5131951c-29b7-4a8c-b221-055451be6bab admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:07.126468 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5131951c-29b7-4a8c-b221-055451be6bab admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:07.127145 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5131951c-29b7-4a8c-b221-055451be6bab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:07.128865 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5131951c-29b7-4a8c-b221-055451be6bab admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:07.143340 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 983/1977] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:43:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:43:07.144240 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69cfb1c-7f1e-4484-9f04-d312cb20ed97 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:07.145413 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5131951c-29b7-4a8c-b221-055451be6bab admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:07.146444 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-b69cfb1c-7f1e-4484-9f04-d312cb20ed97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:07.156044 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5131951c-29b7-4a8c-b221-055451be6bab admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:07.164676 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd04691-93fa-4f74-b052-c8887c725d66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:07.164676 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd04691-93fa-4f74-b052-c8887c725d66 admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:07.164676 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd04691-93fa-4f74-b052-c8887c725d66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:07.164676 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd04691-93fa-4f74-b052-c8887c725d66 admin admin] PATH_INFO: `/v3/projects/c743ba4f164b40d797b53008cfae2aaf/users/c3ad8fcc1f184a2cbca4fd75fef27a11/roles/44b64db484f24a46bf0b7ced46b519bf` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:07.171057 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69cfb1c-7f1e-4484-9f04-d312cb20ed97 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:07.178706 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cb648d-03cf-40e0-9bf6-01c5caae2a95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:07.178706 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cb648d-03cf-40e0-9bf6-01c5caae2a95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:07.178706 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cb648d-03cf-40e0-9bf6-01c5caae2a95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:07.178994 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cb648d-03cf-40e0-9bf6-01c5caae2a95 admin admin] PATH_INFO: `/v3/projects/fd8c6e2240a94ac8b156236aecaf3b88` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:07.180324 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cb648d-03cf-40e0-9bf6-01c5caae2a95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd8c6e2240a94ac8b156236aecaf3b88)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:07.182583 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cb648d-03cf-40e0-9bf6-01c5caae2a95 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:07.182769 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd04691-93fa-4f74-b052-c8887c725d66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c743ba4f164b40d797b53008cfae2aaf, user_id=c3ad8fcc1f184a2cbca4fd75fef27a11, role_id=44b64db484f24a46bf0b7ced46b519bf)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:07.185863 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd04691-93fa-4f74-b052-c8887c725d66 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:07.229848 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 984/1978] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 08:43:07 2022] PUT /identity/v3/projects/c743ba4f164b40d797b53008cfae2aaf/users/c3ad8fcc1f184a2cbca4fd75fef27a11/roles/44b64db484f24a46bf0b7ced46b519bf => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:43:07.240825 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a484a94b-c64b-4ec9-832f-0a64122cfaed None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:07.241141 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a484a94b-c64b-4ec9-832f-0a64122cfaed None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:07.241415 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a484a94b-c64b-4ec9-832f-0a64122cfaed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:07.270367 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-a484a94b-c64b-4ec9-832f-0a64122cfaed None None] MFA Rules not processed for user `c3ad8fcc1f184a2cbca4fd75fef27a11`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:43:07.271564 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-a484a94b-c64b-4ec9-832f-0a64122cfaed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:07.295225 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-c2cb648d-03cf-40e0-9bf6-01c5caae2a95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd8c6e2240a94ac8b156236aecaf3b88', 'request_id': 'req-c2cb648d-03cf-40e0-9bf6-01c5caae2a95'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:07.302823 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-c2cb648d-03cf-40e0-9bf6-01c5caae2a95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd8c6e2240a94ac8b156236aecaf3b88', 'request_id': 'req-c2cb648d-03cf-40e0-9bf6-01c5caae2a95'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:07.310782 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=86,keystone:DELETE=9,keystone:INSERT=5 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:43:07.343779 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 985/1979] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:43:07 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 08:43:07.349241 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a484a94b-c64b-4ec9-832f-0a64122cfaed None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:07.382583 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-c2cb648d-03cf-40e0-9bf6-01c5caae2a95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd8c6e2240a94ac8b156236aecaf3b88', 'request_id': 'req-c2cb648d-03cf-40e0-9bf6-01c5caae2a95'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:07.400572 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-c2cb648d-03cf-40e0-9bf6-01c5caae2a95 admin admin] The token cache is being invalidate because project fd8c6e2240a94ac8b156236aecaf3b88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 08:43:07.400861 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-c2cb648d-03cf-40e0-9bf6-01c5caae2a95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:07.402757 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-c2cb648d-03cf-40e0-9bf6-01c5caae2a95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:07.413094 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 995/1980] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 08:43:07 2022] DELETE /identity/v3/projects/fd8c6e2240a94ac8b156236aecaf3b88 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 08:43:07.414656 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10e5bb8-04be-4027-8ff1-d36213f40fc8 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:07.414946 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10e5bb8-04be-4027-8ff1-d36213f40fc8 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:07.415219 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10e5bb8-04be-4027-8ff1-d36213f40fc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:07.444680 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-d10e5bb8-04be-4027-8ff1-d36213f40fc8 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:43:07.445944 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-d10e5bb8-04be-4027-8ff1-d36213f40fc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:07.446447 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a484a94b-c64b-4ec9-832f-0a64122cfaed None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:07.515150 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e31119b-bea5-472d-941a-ccfa3bc306e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:07.515880 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e31119b-bea5-472d-941a-ccfa3bc306e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:07.516157 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e31119b-bea5-472d-941a-ccfa3bc306e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:07.516428 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e31119b-bea5-472d-941a-ccfa3bc306e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:07.517083 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e31119b-bea5-472d-941a-ccfa3bc306e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:07.519043 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e31119b-bea5-472d-941a-ccfa3bc306e6 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:07.528931 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 996/1981] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:43:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:43:07.530586 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2875b7-c303-4d3c-a0a4-afa14a5a7312 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:07.530733 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2875b7-c303-4d3c-a0a4-afa14a5a7312 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:07.531016 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2875b7-c303-4d3c-a0a4-afa14a5a7312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:07.551907 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-2b2875b7-c303-4d3c-a0a4-afa14a5a7312 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:43:07.553387 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-2b2875b7-c303-4d3c-a0a4-afa14a5a7312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:07.561400 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 997/1982] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:43:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:43:07.565564 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2875b7-c303-4d3c-a0a4-afa14a5a7312 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:07.575760 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2875b7-c303-4d3c-a0a4-afa14a5a7312 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:07.580422 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e3061b-ac94-4f11-b054-9afc088e9a68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:07.581041 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e3061b-ac94-4f11-b054-9afc088e9a68 admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:07.581342 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e3061b-ac94-4f11-b054-9afc088e9a68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:07.581616 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e3061b-ac94-4f11-b054-9afc088e9a68 admin admin] PATH_INFO: `/v3/domains` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:07.582298 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e3061b-ac94-4f11-b054-9afc088e9a68 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:43:07.582602 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e3061b-ac94-4f11-b054-9afc088e9a68 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:07.584278 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e3061b-ac94-4f11-b054-9afc088e9a68 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:07.596180 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 998/1983] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Aug 10 08:43:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:43:07.600474 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e3061b-ac94-4f11-b054-9afc088e9a68 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:07.609989 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e3061b-ac94-4f11-b054-9afc088e9a68 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:07.615305 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed6264c-dc08-4947-833b-28b237778acc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:07.615908 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed6264c-dc08-4947-833b-28b237778acc admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:07.616306 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed6264c-dc08-4947-833b-28b237778acc admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:07.616507 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed6264c-dc08-4947-833b-28b237778acc admin admin] PATH_INFO: `/v3/projects` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:07.617111 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed6264c-dc08-4947-833b-28b237778acc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:07.619009 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed6264c-dc08-4947-833b-28b237778acc admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:07.627757 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 986/1984] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 08:43:07 2022] POST /identity/v3/projects => generated 365 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 08:43:07.628742 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e31119b-bea5-472d-941a-ccfa3bc306e6 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:07.630247 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:43:07.631339 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-2e31119b-bea5-472d-941a-ccfa3bc306e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:07.692862 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 999/1985] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 08:43:07 2022] POST /identity/v3/projects => generated 393 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 08:43:07.698180 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed6264c-dc08-4947-833b-28b237778acc admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:07.714203 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e31119b-bea5-472d-941a-ccfa3bc306e6 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:07.723454 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:43:07.772315 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed6264c-dc08-4947-833b-28b237778acc admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:07.777371 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0b24c5-9dba-45d5-aa82-5725be7d6d5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:07.778005 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0b24c5-9dba-45d5-aa82-5725be7d6d5e admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:07.779655 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0b24c5-9dba-45d5-aa82-5725be7d6d5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:07.779655 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0b24c5-9dba-45d5-aa82-5725be7d6d5e admin admin] PATH_INFO: `/v3/users` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:07.779655 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0b24c5-9dba-45d5-aa82-5725be7d6d5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:07.781317 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0b24c5-9dba-45d5-aa82-5725be7d6d5e admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:07.786412 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503ec7a9-543a-494c-b422-75546004186a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:07.787047 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503ec7a9-543a-494c-b422-75546004186a admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:07.787394 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503ec7a9-543a-494c-b422-75546004186a admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:07.787729 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503ec7a9-543a-494c-b422-75546004186a admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:07.788439 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503ec7a9-543a-494c-b422-75546004186a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:07.790795 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503ec7a9-543a-494c-b422-75546004186a admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:07.869963 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1000/1986] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 08:43:07 2022] POST /identity/v3/users => generated 297 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 08:43:07.874369 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0b24c5-9dba-45d5-aa82-5725be7d6d5e admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:07.885464 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0b24c5-9dba-45d5-aa82-5725be7d6d5e admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:07.890259 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f90b10-4230-44f5-8b55-791007fd5e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:07.890885 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f90b10-4230-44f5-8b55-791007fd5e8c admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:07.891168 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f90b10-4230-44f5-8b55-791007fd5e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:07.891445 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f90b10-4230-44f5-8b55-791007fd5e8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:07.892232 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f90b10-4230-44f5-8b55-791007fd5e8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:07.893994 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f90b10-4230-44f5-8b55-791007fd5e8c admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:07.898935 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 987/1987] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 08:43:07 2022] POST /identity/v3/users => generated 289 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 08:43:07.903042 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503ec7a9-543a-494c-b422-75546004186a admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:07.907163 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1001/1988] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:43:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 08:43:07.912077 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f90b10-4230-44f5-8b55-791007fd5e8c admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:07.914288 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503ec7a9-543a-494c-b422-75546004186a admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:07.919360 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b557a3ba-0e5a-464e-8fcc-5c8c0f7a1715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:07.920003 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b557a3ba-0e5a-464e-8fcc-5c8c0f7a1715 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:07.920318 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b557a3ba-0e5a-464e-8fcc-5c8c0f7a1715 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:07.920557 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b557a3ba-0e5a-464e-8fcc-5c8c0f7a1715 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:07.921252 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b557a3ba-0e5a-464e-8fcc-5c8c0f7a1715 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:07.922860 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b557a3ba-0e5a-464e-8fcc-5c8c0f7a1715 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:07.924128 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f90b10-4230-44f5-8b55-791007fd5e8c admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:07.930207 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedb83f4-2c0b-4289-a5c6-a2a1fa4b8983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:07.932374 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedb83f4-2c0b-4289-a5c6-a2a1fa4b8983 admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:07.932374 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedb83f4-2c0b-4289-a5c6-a2a1fa4b8983 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:07.932374 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedb83f4-2c0b-4289-a5c6-a2a1fa4b8983 admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:07.932374 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedb83f4-2c0b-4289-a5c6-a2a1fa4b8983 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:07.934087 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedb83f4-2c0b-4289-a5c6-a2a1fa4b8983 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:07.936796 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 988/1989] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:43:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 08:43:07.944517 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b557a3ba-0e5a-464e-8fcc-5c8c0f7a1715 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:07.947396 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1002/1990] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:43:07 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:43:07.951628 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedb83f4-2c0b-4289-a5c6-a2a1fa4b8983 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:07.955407 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b557a3ba-0e5a-464e-8fcc-5c8c0f7a1715 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:07.961220 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220807e5-bf4b-4d58-9126-4336db18b661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:07.961887 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220807e5-bf4b-4d58-9126-4336db18b661 admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:07.962162 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220807e5-bf4b-4d58-9126-4336db18b661 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:07.962442 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220807e5-bf4b-4d58-9126-4336db18b661 admin admin] PATH_INFO: `/v3/projects/4d2a140dc7e24645a7b14f6a3e105eec/users/d2680c0d40b443eda0338da2415a3955/roles/ec3751692d9c4ecf81b4a71c936c46be` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:07.962599 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedb83f4-2c0b-4289-a5c6-a2a1fa4b8983 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:07.968753 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdb972c-a59d-4376-9751-0c66b26817d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:07.969536 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdb972c-a59d-4376-9751-0c66b26817d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:07.969905 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdb972c-a59d-4376-9751-0c66b26817d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:07.970247 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdb972c-a59d-4376-9751-0c66b26817d3 admin admin] PATH_INFO: `/v3/projects/2812ed7b559544548cc0c6bfcea6f614/users/809357afea274946bf856abf1f33ce39/roles/44b64db484f24a46bf0b7ced46b519bf` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:07.983612 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220807e5-bf4b-4d58-9126-4336db18b661 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d2a140dc7e24645a7b14f6a3e105eec, user_id=d2680c0d40b443eda0338da2415a3955, role_id=ec3751692d9c4ecf81b4a71c936c46be)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:07.986796 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220807e5-bf4b-4d58-9126-4336db18b661 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:07.987202 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdb972c-a59d-4376-9751-0c66b26817d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2812ed7b559544548cc0c6bfcea6f614, user_id=809357afea274946bf856abf1f33ce39, role_id=44b64db484f24a46bf0b7ced46b519bf)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:07.990060 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdb972c-a59d-4376-9751-0c66b26817d3 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:08.047724 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 989/1991] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 08:43:07 2022] PUT /identity/v3/projects/4d2a140dc7e24645a7b14f6a3e105eec/users/d2680c0d40b443eda0338da2415a3955/roles/ec3751692d9c4ecf81b4a71c936c46be => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 08:43:08.051626 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220807e5-bf4b-4d58-9126-4336db18b661 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:08.070260 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1003/1992] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 08:43:07 2022] PUT /identity/v3/projects/2812ed7b559544548cc0c6bfcea6f614/users/809357afea274946bf856abf1f33ce39/roles/44b64db484f24a46bf0b7ced46b519bf => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:43:08.083177 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84168b0a-f17a-45d5-92dd-7f892954d5ce None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:08.083536 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84168b0a-f17a-45d5-92dd-7f892954d5ce None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:08.083902 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84168b0a-f17a-45d5-92dd-7f892954d5ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:08.111399 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-84168b0a-f17a-45d5-92dd-7f892954d5ce None None] MFA Rules not processed for user `809357afea274946bf856abf1f33ce39`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:43:08.112634 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-84168b0a-f17a-45d5-92dd-7f892954d5ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:08.116589 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220807e5-bf4b-4d58-9126-4336db18b661 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:08.187482 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3934c13e-3e62-418d-a843-cfacf6f706f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:08.188204 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3934c13e-3e62-418d-a843-cfacf6f706f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:08.188551 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3934c13e-3e62-418d-a843-cfacf6f706f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:08.188987 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3934c13e-3e62-418d-a843-cfacf6f706f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:08.189689 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3934c13e-3e62-418d-a843-cfacf6f706f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:08.191462 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3934c13e-3e62-418d-a843-cfacf6f706f4 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:08.192710 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1004/1993] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:43:08 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 08:43:08.196574 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84168b0a-f17a-45d5-92dd-7f892954d5ce None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:08.206524 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 990/1994] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:43:08 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 08:43:08.208037 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3934c13e-3e62-418d-a843-cfacf6f706f4 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:08.211161 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84168b0a-f17a-45d5-92dd-7f892954d5ce None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:08.216211 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a108652-1d5d-4b4b-b02d-ada19f07c837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:08.216864 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a108652-1d5d-4b4b-b02d-ada19f07c837 admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:08.217362 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a108652-1d5d-4b4b-b02d-ada19f07c837 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:08.217698 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a108652-1d5d-4b4b-b02d-ada19f07c837 admin admin] PATH_INFO: `/v3/projects` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:08.218388 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a108652-1d5d-4b4b-b02d-ada19f07c837 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:08.220454 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a108652-1d5d-4b4b-b02d-ada19f07c837 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:08.221576 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3934c13e-3e62-418d-a843-cfacf6f706f4 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:08.226899 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa55bb0c-a33b-437c-91cb-735879bab8b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:08.227583 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa55bb0c-a33b-437c-91cb-735879bab8b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:08.227863 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa55bb0c-a33b-437c-91cb-735879bab8b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:08.228439 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa55bb0c-a33b-437c-91cb-735879bab8b3 admin admin] PATH_INFO: `/v3/projects/4d2a140dc7e24645a7b14f6a3e105eec/users/d2680c0d40b443eda0338da2415a3955/roles/ec3751692d9c4ecf81b4a71c936c46be` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:08.230093 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa55bb0c-a33b-437c-91cb-735879bab8b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d2a140dc7e24645a7b14f6a3e105eec, user_id=d2680c0d40b443eda0338da2415a3955, role_id=ec3751692d9c4ecf81b4a71c936c46be)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:08.232904 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa55bb0c-a33b-437c-91cb-735879bab8b3 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:08.241018 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:43:08.246389 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 991/1995] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 08:43:08 2022] PUT /identity/v3/projects/4d2a140dc7e24645a7b14f6a3e105eec/users/d2680c0d40b443eda0338da2415a3955/roles/ec3751692d9c4ecf81b4a71c936c46be => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 08:43:08.269520 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a6efa0-6941-4557-8c1c-d4bb003e0701 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:08.269863 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a6efa0-6941-4557-8c1c-d4bb003e0701 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:08.270264 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a6efa0-6941-4557-8c1c-d4bb003e0701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:08.302173 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-d5a6efa0-6941-4557-8c1c-d4bb003e0701 None None] MFA Rules not processed for user `d2680c0d40b443eda0338da2415a3955`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:43:08.303930 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-d5a6efa0-6941-4557-8c1c-d4bb003e0701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:08.343226 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:43:08.357726 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1005/1996] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 08:43:08 2022] POST /identity/v3/projects => generated 393 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:43:08.359357 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a108652-1d5d-4b4b-b02d-ada19f07c837 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:08.431935 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 992/1997] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:43:08 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 08:43:08.445380 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a6efa0-6941-4557-8c1c-d4bb003e0701 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:08.446837 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-d5a6efa0-6941-4557-8c1c-d4bb003e0701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:08.451056 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a108652-1d5d-4b4b-b02d-ada19f07c837 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:08.456249 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7e0256-87e7-49a8-aa19-32380d5d7de4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:08.456873 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7e0256-87e7-49a8-aa19-32380d5d7de4 admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:08.457267 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7e0256-87e7-49a8-aa19-32380d5d7de4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:08.457499 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7e0256-87e7-49a8-aa19-32380d5d7de4 admin admin] PATH_INFO: `/v3/users` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:08.458132 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7e0256-87e7-49a8-aa19-32380d5d7de4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:08.460397 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7e0256-87e7-49a8-aa19-32380d5d7de4 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:08.512105 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a6efa0-6941-4557-8c1c-d4bb003e0701 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:08.545492 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1006/1998] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 08:43:08 2022] POST /identity/v3/users => generated 303 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:43:08.549094 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7e0256-87e7-49a8-aa19-32380d5d7de4 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:08.559454 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7e0256-87e7-49a8-aa19-32380d5d7de4 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:08.566792 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b13a0c0-e72c-4f13-ae0e-42b711db7bb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:08.568036 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b13a0c0-e72c-4f13-ae0e-42b711db7bb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:08.568355 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b13a0c0-e72c-4f13-ae0e-42b711db7bb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:08.568612 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b13a0c0-e72c-4f13-ae0e-42b711db7bb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:08.569352 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b13a0c0-e72c-4f13-ae0e-42b711db7bb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:08.570233 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fad873e-18ec-4417-b8a7-5ce77b472b9b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ba74e28ced4cbba857f54222dfbe41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:08.570985 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fad873e-18ec-4417-b8a7-5ce77b472b9b service cinder] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:08.571192 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b13a0c0-e72c-4f13-ae0e-42b711db7bb8 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:08.571605 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fad873e-18ec-4417-b8a7-5ce77b472b9b service cinder] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:08.571984 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fad873e-18ec-4417-b8a7-5ce77b472b9b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:08.582757 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1007/1999] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:43:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 08:43:08.586407 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b13a0c0-e72c-4f13-ae0e-42b711db7bb8 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:08.596722 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b13a0c0-e72c-4f13-ae0e-42b711db7bb8 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:08.605079 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103d1871-2db2-478f-9699-59156f80570b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:08.605744 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103d1871-2db2-478f-9699-59156f80570b admin admin] REQUEST_METHOD: `PUT` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:08.606026 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103d1871-2db2-478f-9699-59156f80570b admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:08.606295 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103d1871-2db2-478f-9699-59156f80570b admin admin] PATH_INFO: `/v3/projects/2213b1cd46364e8b979abb2cf336fdb9/users/08231f4c41484b70a20c410870312c35/roles/ec3751692d9c4ecf81b4a71c936c46be` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:08.623716 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103d1871-2db2-478f-9699-59156f80570b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2213b1cd46364e8b979abb2cf336fdb9, user_id=08231f4c41484b70a20c410870312c35, role_id=ec3751692d9c4ecf81b4a71c936c46be)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:08.626602 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103d1871-2db2-478f-9699-59156f80570b admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:08.640474 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:43:08.646823 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fad873e-18ec-4417-b8a7-5ce77b472b9b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:08.648245 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fad873e-18ec-4417-b8a7-5ce77b472b9b service cinder] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:08.657631 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 993/2000] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:43:08 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 08:43:08.703359 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1008/2001] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 08:43:08 2022] PUT /identity/v3/projects/2213b1cd46364e8b979abb2cf336fdb9/users/08231f4c41484b70a20c410870312c35/roles/ec3751692d9c4ecf81b4a71c936c46be => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:43:08.708621 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fad873e-18ec-4417-b8a7-5ce77b472b9b service cinder] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:08.727728 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fad873e-18ec-4417-b8a7-5ce77b472b9b service cinder] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:08.786738 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a713690-aac1-4467-8254-6d38290ff750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:08.787982 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a713690-aac1-4467-8254-6d38290ff750 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:08.787982 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a713690-aac1-4467-8254-6d38290ff750 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:08.787982 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a713690-aac1-4467-8254-6d38290ff750 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:08.788668 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a713690-aac1-4467-8254-6d38290ff750 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:08.790358 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a713690-aac1-4467-8254-6d38290ff750 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:08.804077 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 994/2002] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:43:08 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:43:08.805239 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103d1871-2db2-478f-9699-59156f80570b admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:08.815960 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103d1871-2db2-478f-9699-59156f80570b admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:08.821229 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c015ef5d-d1e0-4d1c-b24d-35e61bf3832a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:08.821921 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c015ef5d-d1e0-4d1c-b24d-35e61bf3832a admin admin] REQUEST_METHOD: `PUT` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:08.822201 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c015ef5d-d1e0-4d1c-b24d-35e61bf3832a admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:08.822472 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c015ef5d-d1e0-4d1c-b24d-35e61bf3832a admin admin] PATH_INFO: `/v3/projects/2213b1cd46364e8b979abb2cf336fdb9/users/08231f4c41484b70a20c410870312c35/roles/ec3751692d9c4ecf81b4a71c936c46be` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:08.824362 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c015ef5d-d1e0-4d1c-b24d-35e61bf3832a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2213b1cd46364e8b979abb2cf336fdb9, user_id=08231f4c41484b70a20c410870312c35, role_id=ec3751692d9c4ecf81b4a71c936c46be)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:08.827098 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c015ef5d-d1e0-4d1c-b24d-35e61bf3832a admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:08.837556 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1009/2003] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 08:43:08 2022] PUT /identity/v3/projects/2213b1cd46364e8b979abb2cf336fdb9/users/08231f4c41484b70a20c410870312c35/roles/ec3751692d9c4ecf81b4a71c936c46be => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:43:08.849764 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dd8df8-78c4-4984-9097-92229b6720d7 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:08.850050 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dd8df8-78c4-4984-9097-92229b6720d7 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:08.850328 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dd8df8-78c4-4984-9097-92229b6720d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:08.882284 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c015ef5d-d1e0-4d1c-b24d-35e61bf3832a admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:08.884938 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-33dd8df8-78c4-4984-9097-92229b6720d7 None None] MFA Rules not processed for user `08231f4c41484b70a20c410870312c35`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:43:08.886148 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-33dd8df8-78c4-4984-9097-92229b6720d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:08.941215 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c015ef5d-d1e0-4d1c-b24d-35e61bf3832a admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:08.945625 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16613c09-21b6-4bf8-af4f-04a0c6743651 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ba74e28ced4cbba857f54222dfbe41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:08.946254 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16613c09-21b6-4bf8-af4f-04a0c6743651 service cinder] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:08.946517 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16613c09-21b6-4bf8-af4f-04a0c6743651 service cinder] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:08.946785 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16613c09-21b6-4bf8-af4f-04a0c6743651 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:08.948516 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-16613c09-21b6-4bf8-af4f-04a0c6743651 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:08.958745 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 995/2004] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:43:08 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:43:08.969691 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dd8df8-78c4-4984-9097-92229b6720d7 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:08.971046 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-33dd8df8-78c4-4984-9097-92229b6720d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:08.983589 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dd8df8-78c4-4984-9097-92229b6720d7 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:08.986586 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8949c1-60d9-4225-8022-24726c7afda3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ba74e28ced4cbba857f54222dfbe41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:08.987154 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8949c1-60d9-4225-8022-24726c7afda3 service cinder] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:08.987436 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8949c1-60d9-4225-8022-24726c7afda3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:08.987708 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8949c1-60d9-4225-8022-24726c7afda3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:08.997936 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8949c1-60d9-4225-8022-24726c7afda3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:08.999347 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8949c1-60d9-4225-8022-24726c7afda3 service cinder] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:09.008257 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:43:09.010704 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16613c09-21b6-4bf8-af4f-04a0c6743651 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:09.012146 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16613c09-21b6-4bf8-af4f-04a0c6743651 service cinder] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:09.012659 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 996/2005] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:43:08 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 08:43:09.022943 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1010/2006] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:43:08 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 08:43:09.070607 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8949c1-60d9-4225-8022-24726c7afda3 service cinder] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:09.079683 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8949c1-60d9-4225-8022-24726c7afda3 service cinder] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:09.086262 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500bbb66-1833-4c99-8c96-aa68e8ba056a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ba74e28ced4cbba857f54222dfbe41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:09.086935 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500bbb66-1833-4c99-8c96-aa68e8ba056a service cinder] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:09.087157 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500bbb66-1833-4c99-8c96-aa68e8ba056a service cinder] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:09.087411 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500bbb66-1833-4c99-8c96-aa68e8ba056a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:09.162652 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:43:09.165251 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500bbb66-1833-4c99-8c96-aa68e8ba056a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:09.166849 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500bbb66-1833-4c99-8c96-aa68e8ba056a service cinder] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:09.177805 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 997/2007] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:43:09 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:43:12.816078 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16613c09-21b6-4bf8-af4f-04a0c6743651 service cinder] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:12.818858 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-16613c09-21b6-4bf8-af4f-04a0c6743651 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:12.913918 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16613c09-21b6-4bf8-af4f-04a0c6743651 service cinder] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:12.918726 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92cd9d0-9323-4169-9cea-3fae11d9311f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:12.919414 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92cd9d0-9323-4169-9cea-3fae11d9311f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:12.919773 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92cd9d0-9323-4169-9cea-3fae11d9311f admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:12.919943 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92cd9d0-9323-4169-9cea-3fae11d9311f admin admin] PATH_INFO: `/v3/users/534a012969ef4493868aee2737da0a74` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:12.921121 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92cd9d0-9323-4169-9cea-3fae11d9311f admin admin] RBAC: Authorizing `identity:delete_user(user_id=534a012969ef4493868aee2737da0a74)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:12.923058 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92cd9d0-9323-4169-9cea-3fae11d9311f admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:13.113995 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-f92cd9d0-9323-4169-9cea-3fae11d9311f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '534a012969ef4493868aee2737da0a74'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:13.132738 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-f92cd9d0-9323-4169-9cea-3fae11d9311f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '534a012969ef4493868aee2737da0a74'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:13.149511 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-f92cd9d0-9323-4169-9cea-3fae11d9311f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '534a012969ef4493868aee2737da0a74'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:13.222906 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-f92cd9d0-9323-4169-9cea-3fae11d9311f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '534a012969ef4493868aee2737da0a74'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:13.223634 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-f92cd9d0-9323-4169-9cea-3fae11d9311f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '534a012969ef4493868aee2737da0a74'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:13.231796 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1011/2008] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 08:43:12 2022] DELETE /identity/v3/users/534a012969ef4493868aee2737da0a74 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:43:13.236477 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50882c96-d5d8-49e1-bfdd-60039aee7909 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:13.237014 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50882c96-d5d8-49e1-bfdd-60039aee7909 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:13.237277 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50882c96-d5d8-49e1-bfdd-60039aee7909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:13.264191 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-50882c96-d5d8-49e1-bfdd-60039aee7909 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:43:13.266535 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-50882c96-d5d8-49e1-bfdd-60039aee7909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:13.370121 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 998/2009] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:43:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:43:13.385195 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92cd9d0-9323-4169-9cea-3fae11d9311f admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:13.385195 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-f92cd9d0-9323-4169-9cea-3fae11d9311f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:13.460770 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92cd9d0-9323-4169-9cea-3fae11d9311f admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:13.466074 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8c2228-b3ac-41c2-8e03-8301a0f31643 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:13.466767 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8c2228-b3ac-41c2-8e03-8301a0f31643 service neutron] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:13.467143 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8c2228-b3ac-41c2-8e03-8301a0f31643 service neutron] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:13.467488 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8c2228-b3ac-41c2-8e03-8301a0f31643 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:13.477914 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8c2228-b3ac-41c2-8e03-8301a0f31643 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:13.479543 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8c2228-b3ac-41c2-8e03-8301a0f31643 service neutron] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:13.492677 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1012/2010] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:43:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:43:14.155868 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50882c96-d5d8-49e1-bfdd-60039aee7909 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:14.158142 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-50882c96-d5d8-49e1-bfdd-60039aee7909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:14.176280 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50882c96-d5d8-49e1-bfdd-60039aee7909 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:14.185206 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0f5a27-3b20-4a3a-8cd7-a32d638c1c68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:14.185970 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0f5a27-3b20-4a3a-8cd7-a32d638c1c68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:14.186391 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0f5a27-3b20-4a3a-8cd7-a32d638c1c68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:14.186670 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0f5a27-3b20-4a3a-8cd7-a32d638c1c68 admin admin] PATH_INFO: `/v3/projects/1ec81b012136413ba8fb9a9f61ca6625` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:14.197967 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0f5a27-3b20-4a3a-8cd7-a32d638c1c68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ec81b012136413ba8fb9a9f61ca6625)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:14.200686 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0f5a27-3b20-4a3a-8cd7-a32d638c1c68 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:14.346829 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-ea0f5a27-3b20-4a3a-8cd7-a32d638c1c68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ec81b012136413ba8fb9a9f61ca6625', 'request_id': 'req-ea0f5a27-3b20-4a3a-8cd7-a32d638c1c68'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:14.479548 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-ea0f5a27-3b20-4a3a-8cd7-a32d638c1c68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ec81b012136413ba8fb9a9f61ca6625', 'request_id': 'req-ea0f5a27-3b20-4a3a-8cd7-a32d638c1c68'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:14.489706 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-ea0f5a27-3b20-4a3a-8cd7-a32d638c1c68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ec81b012136413ba8fb9a9f61ca6625', 'request_id': 'req-ea0f5a27-3b20-4a3a-8cd7-a32d638c1c68'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:14.519379 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-ea0f5a27-3b20-4a3a-8cd7-a32d638c1c68 admin admin] The token cache is being invalidate because project 1ec81b012136413ba8fb9a9f61ca6625 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 08:43:14.520045 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-ea0f5a27-3b20-4a3a-8cd7-a32d638c1c68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:14.521678 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-ea0f5a27-3b20-4a3a-8cd7-a32d638c1c68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:14.539384 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 999/2011] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 08:43:14 2022] DELETE /identity/v3/projects/1ec81b012136413ba8fb9a9f61ca6625 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:43:16.126189 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8c2228-b3ac-41c2-8e03-8301a0f31643 service neutron] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:16.127739 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-5c8c2228-b3ac-41c2-8e03-8301a0f31643 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:16.141786 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8c2228-b3ac-41c2-8e03-8301a0f31643 service neutron] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:16.159746 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:43:16.204732 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344a2260-ae00-411b-b52e-8c2778bd7d5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:16.205407 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344a2260-ae00-411b-b52e-8c2778bd7d5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:16.205685 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344a2260-ae00-411b-b52e-8c2778bd7d5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:16.205959 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344a2260-ae00-411b-b52e-8c2778bd7d5a admin admin] PATH_INFO: `/v3/users/809357afea274946bf856abf1f33ce39` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:16.207136 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344a2260-ae00-411b-b52e-8c2778bd7d5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=809357afea274946bf856abf1f33ce39)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:16.209202 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344a2260-ae00-411b-b52e-8c2778bd7d5a admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:16.413671 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-344a2260-ae00-411b-b52e-8c2778bd7d5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '809357afea274946bf856abf1f33ce39'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:16.426048 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-344a2260-ae00-411b-b52e-8c2778bd7d5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '809357afea274946bf856abf1f33ce39'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:16.432792 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-344a2260-ae00-411b-b52e-8c2778bd7d5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '809357afea274946bf856abf1f33ce39'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:16.525852 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-344a2260-ae00-411b-b52e-8c2778bd7d5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '809357afea274946bf856abf1f33ce39'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:16.526486 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-344a2260-ae00-411b-b52e-8c2778bd7d5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '809357afea274946bf856abf1f33ce39'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:16.533108 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1013/2012] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 08:43:16 2022] DELETE /identity/v3/users/809357afea274946bf856abf1f33ce39 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:43:16.538185 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3b17ec-54bf-4ff6-a843-be893b8adad5 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:16.538622 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3b17ec-54bf-4ff6-a843-be893b8adad5 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:16.538724 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3b17ec-54bf-4ff6-a843-be893b8adad5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:16.557683 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-ea3b17ec-54bf-4ff6-a843-be893b8adad5 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:43:16.569170 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-ea3b17ec-54bf-4ff6-a843-be893b8adad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:16.619898 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1000/2013] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:43:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:43:16.629963 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344a2260-ae00-411b-b52e-8c2778bd7d5a admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:16.631916 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-344a2260-ae00-411b-b52e-8c2778bd7d5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:16.690071 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344a2260-ae00-411b-b52e-8c2778bd7d5a admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:16.693747 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7e7f90-737d-4431-9096-aa10924352be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:16.694359 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7e7f90-737d-4431-9096-aa10924352be service neutron] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:16.694641 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7e7f90-737d-4431-9096-aa10924352be service neutron] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:16.694918 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7e7f90-737d-4431-9096-aa10924352be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:16.704128 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7e7f90-737d-4431-9096-aa10924352be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:16.705624 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7e7f90-737d-4431-9096-aa10924352be service neutron] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:16.715567 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1014/2014] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:43:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:43:17.172884 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3b17ec-54bf-4ff6-a843-be893b8adad5 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:17.174370 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-ea3b17ec-54bf-4ff6-a843-be893b8adad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:17.190499 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3b17ec-54bf-4ff6-a843-be893b8adad5 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:17.194600 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7e7f90-737d-4431-9096-aa10924352be service neutron] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:17.198310 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-4a7e7f90-737d-4431-9096-aa10924352be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:17.198707 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9658593b-c354-4942-a76f-e737ea8e03b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:17.198707 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9658593b-c354-4942-a76f-e737ea8e03b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:17.199293 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9658593b-c354-4942-a76f-e737ea8e03b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:17.199714 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9658593b-c354-4942-a76f-e737ea8e03b9 admin admin] PATH_INFO: `/v3/projects/2812ed7b559544548cc0c6bfcea6f614` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:17.203131 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9658593b-c354-4942-a76f-e737ea8e03b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2812ed7b559544548cc0c6bfcea6f614)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:17.205349 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9658593b-c354-4942-a76f-e737ea8e03b9 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:17.221270 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7e7f90-737d-4431-9096-aa10924352be service neutron] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:17.226474 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93facb8-6195-49bc-8a09-9651fa962745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:17.227096 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93facb8-6195-49bc-8a09-9651fa962745 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:17.227386 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93facb8-6195-49bc-8a09-9651fa962745 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:17.232045 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93facb8-6195-49bc-8a09-9651fa962745 admin admin] PATH_INFO: `/v3/users/c3ad8fcc1f184a2cbca4fd75fef27a11` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:17.232263 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:43:17.232379 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93facb8-6195-49bc-8a09-9651fa962745 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3ad8fcc1f184a2cbca4fd75fef27a11)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:17.232379 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93facb8-6195-49bc-8a09-9651fa962745 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:17.314891 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-9658593b-c354-4942-a76f-e737ea8e03b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2812ed7b559544548cc0c6bfcea6f614', 'request_id': 'req-9658593b-c354-4942-a76f-e737ea8e03b9'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:17.406000 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-9658593b-c354-4942-a76f-e737ea8e03b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2812ed7b559544548cc0c6bfcea6f614', 'request_id': 'req-9658593b-c354-4942-a76f-e737ea8e03b9'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:17.410844 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:43:17.412592 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-9658593b-c354-4942-a76f-e737ea8e03b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2812ed7b559544548cc0c6bfcea6f614', 'request_id': 'req-9658593b-c354-4942-a76f-e737ea8e03b9'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:17.419848 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-e93facb8-6195-49bc-8a09-9651fa962745 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3ad8fcc1f184a2cbca4fd75fef27a11'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:17.424554 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-9658593b-c354-4942-a76f-e737ea8e03b9 admin admin] The token cache is being invalidate because project 2812ed7b559544548cc0c6bfcea6f614 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 08:43:17.424849 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-9658593b-c354-4942-a76f-e737ea8e03b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:17.425526 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-9658593b-c354-4942-a76f-e737ea8e03b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:17.427756 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1001/2015] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 08:43:17 2022] DELETE /identity/v3/projects/2812ed7b559544548cc0c6bfcea6f614 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:43:17.431603 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9658593b-c354-4942-a76f-e737ea8e03b9 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:17.432822 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-e93facb8-6195-49bc-8a09-9651fa962745 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3ad8fcc1f184a2cbca4fd75fef27a11'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:17.432984 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-9658593b-c354-4942-a76f-e737ea8e03b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:17.444798 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-e93facb8-6195-49bc-8a09-9651fa962745 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3ad8fcc1f184a2cbca4fd75fef27a11'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:17.498195 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9658593b-c354-4942-a76f-e737ea8e03b9 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:17.502814 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d26a613-448d-444b-a581-c41982ac299a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:17.503428 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d26a613-448d-444b-a581-c41982ac299a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:17.503708 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d26a613-448d-444b-a581-c41982ac299a admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:17.503979 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d26a613-448d-444b-a581-c41982ac299a admin admin] PATH_INFO: `/v3/users/08231f4c41484b70a20c410870312c35` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:17.505143 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d26a613-448d-444b-a581-c41982ac299a admin admin] RBAC: Authorizing `identity:delete_user(user_id=08231f4c41484b70a20c410870312c35)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:17.507057 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d26a613-448d-444b-a581-c41982ac299a admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:17.537858 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-e93facb8-6195-49bc-8a09-9651fa962745 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3ad8fcc1f184a2cbca4fd75fef27a11'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:17.538501 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-e93facb8-6195-49bc-8a09-9651fa962745 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3ad8fcc1f184a2cbca4fd75fef27a11'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:17.544952 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1015/2016] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 08:43:17 2022] DELETE /identity/v3/users/c3ad8fcc1f184a2cbca4fd75fef27a11 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:43:17.549178 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2ab3af-6aff-482b-96f6-af35d61978b4 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:17.549466 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2ab3af-6aff-482b-96f6-af35d61978b4 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:17.549737 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2ab3af-6aff-482b-96f6-af35d61978b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:17.570547 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-5a2ab3af-6aff-482b-96f6-af35d61978b4 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:43:17.571767 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-5a2ab3af-6aff-482b-96f6-af35d61978b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:17.629483 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1016/2017] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:43:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 08:43:17.639801 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2ab3af-6aff-482b-96f6-af35d61978b4 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:17.641159 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-5a2ab3af-6aff-482b-96f6-af35d61978b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:17.696658 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2ab3af-6aff-482b-96f6-af35d61978b4 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:17.700290 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-4d26a613-448d-444b-a581-c41982ac299a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08231f4c41484b70a20c410870312c35'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:17.700752 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49d94b8-9b90-473f-a1cd-86563c1b2212 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:17.701117 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49d94b8-9b90-473f-a1cd-86563c1b2212 service neutron] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:17.701482 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49d94b8-9b90-473f-a1cd-86563c1b2212 service neutron] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:17.701766 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49d94b8-9b90-473f-a1cd-86563c1b2212 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:17.712309 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49d94b8-9b90-473f-a1cd-86563c1b2212 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:17.712585 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-4d26a613-448d-444b-a581-c41982ac299a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08231f4c41484b70a20c410870312c35'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:17.713779 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49d94b8-9b90-473f-a1cd-86563c1b2212 service neutron] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:17.719941 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-4d26a613-448d-444b-a581-c41982ac299a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08231f4c41484b70a20c410870312c35'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:17.720691 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-4d26a613-448d-444b-a581-c41982ac299a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08231f4c41484b70a20c410870312c35'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:17.721359 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-4d26a613-448d-444b-a581-c41982ac299a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08231f4c41484b70a20c410870312c35'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:17.723143 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1017/2018] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:43:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 08:43:17.795032 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1002/2019] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 08:43:17 2022] DELETE /identity/v3/users/08231f4c41484b70a20c410870312c35 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 08:43:18.175927 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49d94b8-9b90-473f-a1cd-86563c1b2212 service neutron] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:18.175927 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-b49d94b8-9b90-473f-a1cd-86563c1b2212 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:18.194661 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49d94b8-9b90-473f-a1cd-86563c1b2212 service neutron] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:18.248696 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d26a613-448d-444b-a581-c41982ac299a admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:18.250520 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-4d26a613-448d-444b-a581-c41982ac299a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:18.252410 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:43:18.263220 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc6fca4-b8f7-4e30-8e84-849834d2ecd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:18.263931 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc6fca4-b8f7-4e30-8e84-849834d2ecd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:18.264204 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc6fca4-b8f7-4e30-8e84-849834d2ecd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:18.264530 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc6fca4-b8f7-4e30-8e84-849834d2ecd9 admin admin] PATH_INFO: `/v3/projects/c743ba4f164b40d797b53008cfae2aaf` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:18.266544 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc6fca4-b8f7-4e30-8e84-849834d2ecd9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c743ba4f164b40d797b53008cfae2aaf)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:18.269171 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc6fca4-b8f7-4e30-8e84-849834d2ecd9 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:18.298167 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:43:18.339448 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d26a613-448d-444b-a581-c41982ac299a admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:18.344177 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d82a13-bbc4-4ebe-8e84-6d2b36552d33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:18.344806 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d82a13-bbc4-4ebe-8e84-6d2b36552d33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:18.345118 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d82a13-bbc4-4ebe-8e84-6d2b36552d33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:18.345380 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d82a13-bbc4-4ebe-8e84-6d2b36552d33 admin admin] PATH_INFO: `/v3/projects/2213b1cd46364e8b979abb2cf336fdb9` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:18.346648 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d82a13-bbc4-4ebe-8e84-6d2b36552d33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2213b1cd46364e8b979abb2cf336fdb9)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:18.348702 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d82a13-bbc4-4ebe-8e84-6d2b36552d33 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:18.362511 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-7fc6fca4-b8f7-4e30-8e84-849834d2ecd9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c743ba4f164b40d797b53008cfae2aaf', 'request_id': 'req-7fc6fca4-b8f7-4e30-8e84-849834d2ecd9'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:18.368856 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-7fc6fca4-b8f7-4e30-8e84-849834d2ecd9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c743ba4f164b40d797b53008cfae2aaf', 'request_id': 'req-7fc6fca4-b8f7-4e30-8e84-849834d2ecd9'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:18.439006 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-7fc6fca4-b8f7-4e30-8e84-849834d2ecd9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c743ba4f164b40d797b53008cfae2aaf', 'request_id': 'req-7fc6fca4-b8f7-4e30-8e84-849834d2ecd9'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:18.450892 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-7fc6fca4-b8f7-4e30-8e84-849834d2ecd9 admin admin] The token cache is being invalidate because project c743ba4f164b40d797b53008cfae2aaf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 08:43:18.451120 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-7fc6fca4-b8f7-4e30-8e84-849834d2ecd9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:18.451750 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-7fc6fca4-b8f7-4e30-8e84-849834d2ecd9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:18.453338 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1018/2020] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 08:43:18 2022] DELETE /identity/v3/projects/c743ba4f164b40d797b53008cfae2aaf => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:43:18.457194 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc6fca4-b8f7-4e30-8e84-849834d2ecd9 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:18.458759 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-7fc6fca4-b8f7-4e30-8e84-849834d2ecd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:18.476172 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-e8d82a13-bbc4-4ebe-8e84-6d2b36552d33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2213b1cd46364e8b979abb2cf336fdb9', 'request_id': 'req-e8d82a13-bbc4-4ebe-8e84-6d2b36552d33'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:18.520288 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc6fca4-b8f7-4e30-8e84-849834d2ecd9 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:18.575417 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496610f5-953c-4ed6-bb14-b5e31d7af877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:18.576069 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496610f5-953c-4ed6-bb14-b5e31d7af877 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:18.576348 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496610f5-953c-4ed6-bb14-b5e31d7af877 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:18.576627 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496610f5-953c-4ed6-bb14-b5e31d7af877 admin admin] PATH_INFO: `/v3/users/d2680c0d40b443eda0338da2415a3955` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:18.577788 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496610f5-953c-4ed6-bb14-b5e31d7af877 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2680c0d40b443eda0338da2415a3955)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:18.579736 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496610f5-953c-4ed6-bb14-b5e31d7af877 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:18.588159 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-e8d82a13-bbc4-4ebe-8e84-6d2b36552d33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2213b1cd46364e8b979abb2cf336fdb9', 'request_id': 'req-e8d82a13-bbc4-4ebe-8e84-6d2b36552d33'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:18.594335 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-e8d82a13-bbc4-4ebe-8e84-6d2b36552d33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2213b1cd46364e8b979abb2cf336fdb9', 'request_id': 'req-e8d82a13-bbc4-4ebe-8e84-6d2b36552d33'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:18.606067 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-e8d82a13-bbc4-4ebe-8e84-6d2b36552d33 admin admin] The token cache is being invalidate because project 2213b1cd46364e8b979abb2cf336fdb9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 08:43:18.606380 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-e8d82a13-bbc4-4ebe-8e84-6d2b36552d33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:18.607033 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-e8d82a13-bbc4-4ebe-8e84-6d2b36552d33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:18.608728 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1003/2021] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 08:43:18 2022] DELETE /identity/v3/projects/2213b1cd46364e8b979abb2cf336fdb9 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:43:18.613486 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0286a1-43fd-4be2-91e1-e33da18af2e2 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:18.613773 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0286a1-43fd-4be2-91e1-e33da18af2e2 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:18.614048 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0286a1-43fd-4be2-91e1-e33da18af2e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:18.633654 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-5d0286a1-43fd-4be2-91e1-e33da18af2e2 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:43:18.634852 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-5d0286a1-43fd-4be2-91e1-e33da18af2e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:18.642643 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1004/2022] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:43:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 08:43:18.650673 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a72500-a40e-4ea9-9e95-6c4e59a7c7fa None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:18.650949 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a72500-a40e-4ea9-9e95-6c4e59a7c7fa None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:18.651228 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a72500-a40e-4ea9-9e95-6c4e59a7c7fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:18.670962 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-50a72500-a40e-4ea9-9e95-6c4e59a7c7fa None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:43:18.672192 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-50a72500-a40e-4ea9-9e95-6c4e59a7c7fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:18.680247 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1005/2023] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:43:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 08:43:18.684491 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a72500-a40e-4ea9-9e95-6c4e59a7c7fa None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:18.693722 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a72500-a40e-4ea9-9e95-6c4e59a7c7fa None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:18.698659 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa774495-ce10-4d3e-a07e-f0bf38fd413b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:18.699323 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa774495-ce10-4d3e-a07e-f0bf38fd413b admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:18.699606 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa774495-ce10-4d3e-a07e-f0bf38fd413b admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:18.699877 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa774495-ce10-4d3e-a07e-f0bf38fd413b admin admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:18.700523 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa774495-ce10-4d3e-a07e-f0bf38fd413b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:43:18.707982 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa774495-ce10-4d3e-a07e-f0bf38fd413b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:18.707982 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa774495-ce10-4d3e-a07e-f0bf38fd413b admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:18.714472 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1006/2024] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Aug 10 08:43:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:43:18.722266 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa774495-ce10-4d3e-a07e-f0bf38fd413b admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:18.732876 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa774495-ce10-4d3e-a07e-f0bf38fd413b admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:18.739239 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53113f85-4d46-480f-b404-295d45d764c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:18.742562 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53113f85-4d46-480f-b404-295d45d764c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:18.742562 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53113f85-4d46-480f-b404-295d45d764c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:18.742562 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53113f85-4d46-480f-b404-295d45d764c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:18.742562 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53113f85-4d46-480f-b404-295d45d764c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:18.742986 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53113f85-4d46-480f-b404-295d45d764c3 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:18.743570 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:43:18.752575 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-496610f5-953c-4ed6-bb14-b5e31d7af877 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2680c0d40b443eda0338da2415a3955'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:18.764311 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-496610f5-953c-4ed6-bb14-b5e31d7af877 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2680c0d40b443eda0338da2415a3955'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:18.771551 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-496610f5-953c-4ed6-bb14-b5e31d7af877 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2680c0d40b443eda0338da2415a3955'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:18.845816 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-496610f5-953c-4ed6-bb14-b5e31d7af877 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2680c0d40b443eda0338da2415a3955'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:18.846575 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-496610f5-953c-4ed6-bb14-b5e31d7af877 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2680c0d40b443eda0338da2415a3955'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:18.849362 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1007/2025] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 08:43:18 2022] POST /identity/v3/projects => generated 385 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 08:43:18.850129 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53113f85-4d46-480f-b404-295d45d764c3 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:18.852019 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-53113f85-4d46-480f-b404-295d45d764c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:18.853389 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1019/2026] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 08:43:18 2022] DELETE /identity/v3/users/d2680c0d40b443eda0338da2415a3955 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 08:43:18.919128 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53113f85-4d46-480f-b404-295d45d764c3 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:18.928482 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:43:18.978405 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ea224c-e214-4905-ac9b-452b8a3cb81b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:18.978940 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ea224c-e214-4905-ac9b-452b8a3cb81b admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:18.979219 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ea224c-e214-4905-ac9b-452b8a3cb81b admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:18.979484 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ea224c-e214-4905-ac9b-452b8a3cb81b admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:18.980094 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ea224c-e214-4905-ac9b-452b8a3cb81b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:18.982027 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ea224c-e214-4905-ac9b-452b8a3cb81b admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:19.104466 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1008/2027] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 08:43:18 2022] POST /identity/v3/users => generated 293 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 08:43:19.110092 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496610f5-953c-4ed6-bb14-b5e31d7af877 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:19.122583 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496610f5-953c-4ed6-bb14-b5e31d7af877 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:19.127748 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93527bec-10e0-4e7d-97e7-4947b1b2884b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:19.128388 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93527bec-10e0-4e7d-97e7-4947b1b2884b admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:19.128962 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93527bec-10e0-4e7d-97e7-4947b1b2884b admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:19.128962 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93527bec-10e0-4e7d-97e7-4947b1b2884b admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:19.129610 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93527bec-10e0-4e7d-97e7-4947b1b2884b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:19.131394 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93527bec-10e0-4e7d-97e7-4947b1b2884b admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:19.142147 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1020/2028] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:43:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 08:43:19.146002 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ea224c-e214-4905-ac9b-452b8a3cb81b admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:19.157291 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ea224c-e214-4905-ac9b-452b8a3cb81b admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:19.163273 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a859ba7c-6d2d-4187-8d77-bef733a72071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:19.164074 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a859ba7c-6d2d-4187-8d77-bef733a72071 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:19.164757 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a859ba7c-6d2d-4187-8d77-bef733a72071 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:19.164757 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a859ba7c-6d2d-4187-8d77-bef733a72071 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:19.168841 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a859ba7c-6d2d-4187-8d77-bef733a72071 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:19.168841 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a859ba7c-6d2d-4187-8d77-bef733a72071 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:19.178349 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1009/2029] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:43:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:43:19.182614 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93527bec-10e0-4e7d-97e7-4947b1b2884b admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:19.192772 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93527bec-10e0-4e7d-97e7-4947b1b2884b admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:19.197439 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47da0b6b-5ff7-4337-a8c3-5f28c3515fa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:19.198104 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47da0b6b-5ff7-4337-a8c3-5f28c3515fa0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:19.198389 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47da0b6b-5ff7-4337-a8c3-5f28c3515fa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:19.198678 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47da0b6b-5ff7-4337-a8c3-5f28c3515fa0 admin admin] PATH_INFO: `/v3/projects/731bc09f5ded4e95a6a2b9d1fcf82c1c/users/94be600600904396895e9e7b981ca8d1/roles/44b64db484f24a46bf0b7ced46b519bf` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:19.216740 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47da0b6b-5ff7-4337-a8c3-5f28c3515fa0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=731bc09f5ded4e95a6a2b9d1fcf82c1c, user_id=94be600600904396895e9e7b981ca8d1, role_id=44b64db484f24a46bf0b7ced46b519bf)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:19.219489 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47da0b6b-5ff7-4337-a8c3-5f28c3515fa0 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:19.317617 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a859ba7c-6d2d-4187-8d77-bef733a72071 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:19.323025 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-a859ba7c-6d2d-4187-8d77-bef733a72071 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:19.323166 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1021/2030] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 08:43:19 2022] PUT /identity/v3/projects/731bc09f5ded4e95a6a2b9d1fcf82c1c/users/94be600600904396895e9e7b981ca8d1/roles/44b64db484f24a46bf0b7ced46b519bf => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 08:43:19.330773 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4169f9-9a6e-4920-ba08-20c72c324018 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:19.331058 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4169f9-9a6e-4920-ba08-20c72c324018 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:19.331328 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4169f9-9a6e-4920-ba08-20c72c324018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:19.342041 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a859ba7c-6d2d-4187-8d77-bef733a72071 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:19.368371 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-4a4169f9-9a6e-4920-ba08-20c72c324018 None None] MFA Rules not processed for user `94be600600904396895e9e7b981ca8d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:43:19.369792 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-4a4169f9-9a6e-4920-ba08-20c72c324018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:19.418285 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411cdd9f-a723-40b4-b870-66e8d0486a0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:19.418973 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411cdd9f-a723-40b4-b870-66e8d0486a0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:19.419258 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411cdd9f-a723-40b4-b870-66e8d0486a0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:19.419552 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411cdd9f-a723-40b4-b870-66e8d0486a0a admin admin] PATH_INFO: `/v3/projects/4d2a140dc7e24645a7b14f6a3e105eec` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:19.420719 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411cdd9f-a723-40b4-b870-66e8d0486a0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d2a140dc7e24645a7b14f6a3e105eec)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:19.422718 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411cdd9f-a723-40b4-b870-66e8d0486a0a admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:19.442641 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1022/2031] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:43:19 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:43:19.454865 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4169f9-9a6e-4920-ba08-20c72c324018 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:19.456254 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-4a4169f9-9a6e-4920-ba08-20c72c324018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:19.513078 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4169f9-9a6e-4920-ba08-20c72c324018 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:19.516603 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3972ece2-be9f-4778-961b-475cf6d5d8e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ba74e28ced4cbba857f54222dfbe41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:19.517240 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3972ece2-be9f-4778-961b-475cf6d5d8e3 service cinder] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:19.517519 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3972ece2-be9f-4778-961b-475cf6d5d8e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:19.517789 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3972ece2-be9f-4778-961b-475cf6d5d8e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:19.526538 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3972ece2-be9f-4778-961b-475cf6d5d8e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:19.527955 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3972ece2-be9f-4778-961b-475cf6d5d8e3 service cinder] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:19.537589 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1023/2032] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:43:19 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 08:43:19.549127 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-411cdd9f-a723-40b4-b870-66e8d0486a0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d2a140dc7e24645a7b14f6a3e105eec', 'request_id': 'req-411cdd9f-a723-40b4-b870-66e8d0486a0a'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:19.628167 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-411cdd9f-a723-40b4-b870-66e8d0486a0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d2a140dc7e24645a7b14f6a3e105eec', 'request_id': 'req-411cdd9f-a723-40b4-b870-66e8d0486a0a'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:19.634355 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-411cdd9f-a723-40b4-b870-66e8d0486a0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d2a140dc7e24645a7b14f6a3e105eec', 'request_id': 'req-411cdd9f-a723-40b4-b870-66e8d0486a0a'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:19.647272 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-411cdd9f-a723-40b4-b870-66e8d0486a0a admin admin] The token cache is being invalidate because project 4d2a140dc7e24645a7b14f6a3e105eec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 08:43:19.647567 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-411cdd9f-a723-40b4-b870-66e8d0486a0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:19.650806 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-411cdd9f-a723-40b4-b870-66e8d0486a0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:19.650806 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1010/2033] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 08:43:19 2022] DELETE /identity/v3/projects/4d2a140dc7e24645a7b14f6a3e105eec => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 08:43:19.654784 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ad8f75-e273-4357-8bd5-f354def9e2d6 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:19.655078 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ad8f75-e273-4357-8bd5-f354def9e2d6 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:19.655350 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ad8f75-e273-4357-8bd5-f354def9e2d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:19.675569 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-17ad8f75-e273-4357-8bd5-f354def9e2d6 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:43:19.676763 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-17ad8f75-e273-4357-8bd5-f354def9e2d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:19.681469 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:43:19.736750 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1024/2034] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:43:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:43:19.744692 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346f053b-483d-4e1f-900f-a0ac923f8651 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:19.745182 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346f053b-483d-4e1f-900f-a0ac923f8651 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:19.745275 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346f053b-483d-4e1f-900f-a0ac923f8651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:19.768874 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-346f053b-483d-4e1f-900f-a0ac923f8651 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:43:19.770098 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-346f053b-483d-4e1f-900f-a0ac923f8651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:19.780263 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1011/2035] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:43:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:43:19.784454 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ad8f75-e273-4357-8bd5-f354def9e2d6 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:19.794817 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ad8f75-e273-4357-8bd5-f354def9e2d6 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:19.799468 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637e296d-fc50-483f-b132-e974c0b4083a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:19.800120 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e296d-fc50-483f-b132-e974c0b4083a admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:19.800460 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e296d-fc50-483f-b132-e974c0b4083a admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:19.800707 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e296d-fc50-483f-b132-e974c0b4083a admin admin] PATH_INFO: `/v3/domains` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:19.801505 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637e296d-fc50-483f-b132-e974c0b4083a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:43:19.801806 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637e296d-fc50-483f-b132-e974c0b4083a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:19.803421 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637e296d-fc50-483f-b132-e974c0b4083a admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:19.815280 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1025/2036] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Aug 10 08:43:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:43:19.820314 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346f053b-483d-4e1f-900f-a0ac923f8651 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:19.829902 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346f053b-483d-4e1f-900f-a0ac923f8651 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:19.835709 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efd14a7-8241-4a83-9dbf-c2e8ec251c1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:19.836301 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efd14a7-8241-4a83-9dbf-c2e8ec251c1e admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:19.836579 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efd14a7-8241-4a83-9dbf-c2e8ec251c1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:19.836844 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efd14a7-8241-4a83-9dbf-c2e8ec251c1e admin admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:19.837498 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efd14a7-8241-4a83-9dbf-c2e8ec251c1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:19.839521 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efd14a7-8241-4a83-9dbf-c2e8ec251c1e admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:19.851005 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:43:19.962745 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1012/2037] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 08:43:19 2022] POST /identity/v3/projects => generated 371 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:43:19.966877 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637e296d-fc50-483f-b132-e974c0b4083a admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:19.977162 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637e296d-fc50-483f-b132-e974c0b4083a admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:20.033759 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fb43f1-2106-4945-84e3-fdf948830361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:20.034269 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fb43f1-2106-4945-84e3-fdf948830361 admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:20.034552 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fb43f1-2106-4945-84e3-fdf948830361 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:20.034854 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fb43f1-2106-4945-84e3-fdf948830361 admin admin] PATH_INFO: `/v3/users` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:20.035488 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fb43f1-2106-4945-84e3-fdf948830361 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:20.037789 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fb43f1-2106-4945-84e3-fdf948830361 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:20.154584 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1026/2038] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 08:43:19 2022] POST /identity/v3/users => generated 286 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:43:20.158640 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efd14a7-8241-4a83-9dbf-c2e8ec251c1e admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:20.170290 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efd14a7-8241-4a83-9dbf-c2e8ec251c1e admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:20.174845 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba77d39-25e6-4f5b-8f6e-c361ea90094e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:20.175478 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba77d39-25e6-4f5b-8f6e-c361ea90094e admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:20.175758 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba77d39-25e6-4f5b-8f6e-c361ea90094e admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:20.176028 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba77d39-25e6-4f5b-8f6e-c361ea90094e admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:20.176663 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba77d39-25e6-4f5b-8f6e-c361ea90094e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:20.178491 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba77d39-25e6-4f5b-8f6e-c361ea90094e admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:20.188596 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1013/2039] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:43:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 08:43:20.192321 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fb43f1-2106-4945-84e3-fdf948830361 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:20.202662 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fb43f1-2106-4945-84e3-fdf948830361 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:20.207424 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d84aa7f-76ca-48ec-ab47-9db7d2d67847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:20.208718 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d84aa7f-76ca-48ec-ab47-9db7d2d67847 admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:20.209174 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d84aa7f-76ca-48ec-ab47-9db7d2d67847 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:20.209626 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d84aa7f-76ca-48ec-ab47-9db7d2d67847 admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:20.210272 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d84aa7f-76ca-48ec-ab47-9db7d2d67847 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:20.211852 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d84aa7f-76ca-48ec-ab47-9db7d2d67847 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:20.226664 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1027/2040] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:43:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:43:20.228787 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba77d39-25e6-4f5b-8f6e-c361ea90094e admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:20.240397 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba77d39-25e6-4f5b-8f6e-c361ea90094e admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:20.243716 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99cae6c-c83a-4cb1-ba3f-2757d7b00cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:20.244415 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99cae6c-c83a-4cb1-ba3f-2757d7b00cc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:20.244692 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99cae6c-c83a-4cb1-ba3f-2757d7b00cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:20.244988 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99cae6c-c83a-4cb1-ba3f-2757d7b00cc9 admin admin] PATH_INFO: `/v3/projects/617fe6188d7d48acb614c0793c31afb2/users/961b5628d49d42f48d64d3fb3a7a8656/roles/44b64db484f24a46bf0b7ced46b519bf` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:20.262318 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99cae6c-c83a-4cb1-ba3f-2757d7b00cc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=617fe6188d7d48acb614c0793c31afb2, user_id=961b5628d49d42f48d64d3fb3a7a8656, role_id=44b64db484f24a46bf0b7ced46b519bf)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:20.266745 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99cae6c-c83a-4cb1-ba3f-2757d7b00cc9 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:20.350167 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1014/2041] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 08:43:20 2022] PUT /identity/v3/projects/617fe6188d7d48acb614c0793c31afb2/users/961b5628d49d42f48d64d3fb3a7a8656/roles/44b64db484f24a46bf0b7ced46b519bf => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:43:20.361342 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e934c2-24f7-4b5d-8b46-03c93867d9ad None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:20.361822 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e934c2-24f7-4b5d-8b46-03c93867d9ad None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:20.362194 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e934c2-24f7-4b5d-8b46-03c93867d9ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:20.393513 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-30e934c2-24f7-4b5d-8b46-03c93867d9ad None None] MFA Rules not processed for user `961b5628d49d42f48d64d3fb3a7a8656`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:43:20.394788 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-30e934c2-24f7-4b5d-8b46-03c93867d9ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:20.476947 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1028/2042] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:43:20 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:43:20.489082 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99cae6c-c83a-4cb1-ba3f-2757d7b00cc9 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:20.491666 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-c99cae6c-c83a-4cb1-ba3f-2757d7b00cc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:20.550199 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99cae6c-c83a-4cb1-ba3f-2757d7b00cc9 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:20.554729 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edaea6c-4178-4e29-8372-9add35a8ce15 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ba74e28ced4cbba857f54222dfbe41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:20.555366 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edaea6c-4178-4e29-8372-9add35a8ce15 service cinder] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:20.559686 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edaea6c-4178-4e29-8372-9add35a8ce15 service cinder] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:20.559686 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edaea6c-4178-4e29-8372-9add35a8ce15 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:20.564237 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edaea6c-4178-4e29-8372-9add35a8ce15 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:20.565681 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edaea6c-4178-4e29-8372-9add35a8ce15 service cinder] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:20.574539 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1015/2043] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:43:20 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:43:20.597256 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e934c2-24f7-4b5d-8b46-03c93867d9ad None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:20.603439 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:43:20.664306 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e934c2-24f7-4b5d-8b46-03c93867d9ad None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:20.669041 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7ab63f-9e98-4f95-97a0-89cc31ed2335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:20.669669 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ab63f-9e98-4f95-97a0-89cc31ed2335 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:20.669945 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ab63f-9e98-4f95-97a0-89cc31ed2335 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:20.670214 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ab63f-9e98-4f95-97a0-89cc31ed2335 admin admin] PATH_INFO: `/v3/users/961b5628d49d42f48d64d3fb3a7a8656` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:20.671346 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7ab63f-9e98-4f95-97a0-89cc31ed2335 admin admin] RBAC: Authorizing `identity:delete_user(user_id=961b5628d49d42f48d64d3fb3a7a8656)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:20.673354 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7ab63f-9e98-4f95-97a0-89cc31ed2335 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:20.916228 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-6f7ab63f-9e98-4f95-97a0-89cc31ed2335 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '961b5628d49d42f48d64d3fb3a7a8656'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:20.927884 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-6f7ab63f-9e98-4f95-97a0-89cc31ed2335 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '961b5628d49d42f48d64d3fb3a7a8656'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:20.934419 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-6f7ab63f-9e98-4f95-97a0-89cc31ed2335 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '961b5628d49d42f48d64d3fb3a7a8656'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:21.011533 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-6f7ab63f-9e98-4f95-97a0-89cc31ed2335 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '961b5628d49d42f48d64d3fb3a7a8656'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:21.012600 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-6f7ab63f-9e98-4f95-97a0-89cc31ed2335 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '961b5628d49d42f48d64d3fb3a7a8656'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:21.019125 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1029/2044] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 08:43:20 2022] DELETE /identity/v3/users/961b5628d49d42f48d64d3fb3a7a8656 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:43:21.023690 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad75c07-3738-4a84-89ab-83d91694d85c None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:21.024194 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad75c07-3738-4a84-89ab-83d91694d85c None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:21.024302 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad75c07-3738-4a84-89ab-83d91694d85c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:21.043573 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-9ad75c07-3738-4a84-89ab-83d91694d85c None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:43:21.044811 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-9ad75c07-3738-4a84-89ab-83d91694d85c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:21.105702 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1016/2045] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:43:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:43:21.115766 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7ab63f-9e98-4f95-97a0-89cc31ed2335 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:21.118332 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-6f7ab63f-9e98-4f95-97a0-89cc31ed2335 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:21.174714 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7ab63f-9e98-4f95-97a0-89cc31ed2335 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:21.179162 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5d96c5-c79f-4c23-89ac-eec6f11922cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:21.179766 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5d96c5-c79f-4c23-89ac-eec6f11922cf service neutron] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:21.180045 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5d96c5-c79f-4c23-89ac-eec6f11922cf service neutron] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:21.180314 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5d96c5-c79f-4c23-89ac-eec6f11922cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:21.190351 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5d96c5-c79f-4c23-89ac-eec6f11922cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:21.191779 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5d96c5-c79f-4c23-89ac-eec6f11922cf service neutron] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:21.203027 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1030/2046] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:43:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 08:43:21.620301 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad75c07-3738-4a84-89ab-83d91694d85c None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:21.622645 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-9ad75c07-3738-4a84-89ab-83d91694d85c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:21.636257 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad75c07-3738-4a84-89ab-83d91694d85c None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:21.642717 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3398ad-1f19-4bf4-82d3-71348850e3b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:21.643383 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3398ad-1f19-4bf4-82d3-71348850e3b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:21.643930 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3398ad-1f19-4bf4-82d3-71348850e3b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:21.644232 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3398ad-1f19-4bf4-82d3-71348850e3b8 admin admin] PATH_INFO: `/v3/projects/617fe6188d7d48acb614c0793c31afb2` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:21.646056 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3398ad-1f19-4bf4-82d3-71348850e3b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=617fe6188d7d48acb614c0793c31afb2)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:21.648573 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3398ad-1f19-4bf4-82d3-71348850e3b8 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:21.743949 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-5f3398ad-1f19-4bf4-82d3-71348850e3b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '617fe6188d7d48acb614c0793c31afb2', 'request_id': 'req-5f3398ad-1f19-4bf4-82d3-71348850e3b8'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:21.748204 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:43:21.822717 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-5f3398ad-1f19-4bf4-82d3-71348850e3b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '617fe6188d7d48acb614c0793c31afb2', 'request_id': 'req-5f3398ad-1f19-4bf4-82d3-71348850e3b8'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:21.830064 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-5f3398ad-1f19-4bf4-82d3-71348850e3b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '617fe6188d7d48acb614c0793c31afb2', 'request_id': 'req-5f3398ad-1f19-4bf4-82d3-71348850e3b8'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:21.845054 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-5f3398ad-1f19-4bf4-82d3-71348850e3b8 admin admin] The token cache is being invalidate because project 617fe6188d7d48acb614c0793c31afb2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 08:43:21.845371 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-5f3398ad-1f19-4bf4-82d3-71348850e3b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:21.846060 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-5f3398ad-1f19-4bf4-82d3-71348850e3b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:21.847556 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1017/2047] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 08:43:21 2022] DELETE /identity/v3/projects/617fe6188d7d48acb614c0793c31afb2 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:43:21.853359 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f053312-8a0b-44a7-939a-24460c82906f None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:21.853668 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f053312-8a0b-44a7-939a-24460c82906f None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:21.858554 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f053312-8a0b-44a7-939a-24460c82906f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:21.875716 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-3f053312-8a0b-44a7-939a-24460c82906f None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:43:21.875716 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-3f053312-8a0b-44a7-939a-24460c82906f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:21.895347 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:43:21.935326 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1031/2048] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:43:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:43:21.947576 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fe669d-e517-414e-a906-ddc6b5017b50 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:21.947868 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fe669d-e517-414e-a906-ddc6b5017b50 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:21.948757 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fe669d-e517-414e-a906-ddc6b5017b50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:21.968459 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-47fe669d-e517-414e-a906-ddc6b5017b50 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:43:21.969704 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-47fe669d-e517-414e-a906-ddc6b5017b50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:21.978272 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1018/2049] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:43:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:43:21.982655 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f053312-8a0b-44a7-939a-24460c82906f None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:21.993851 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f053312-8a0b-44a7-939a-24460c82906f None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:21.998935 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06471919-7c5c-402b-8278-fc749fcb160b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:21.999550 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06471919-7c5c-402b-8278-fc749fcb160b admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:21.999819 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06471919-7c5c-402b-8278-fc749fcb160b admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:22.000089 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06471919-7c5c-402b-8278-fc749fcb160b admin admin] PATH_INFO: `/v3/domains` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:22.000733 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06471919-7c5c-402b-8278-fc749fcb160b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:43:22.001072 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06471919-7c5c-402b-8278-fc749fcb160b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:22.002799 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06471919-7c5c-402b-8278-fc749fcb160b admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:22.015258 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1032/2050] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Aug 10 08:43:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:43:22.020772 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fe669d-e517-414e-a906-ddc6b5017b50 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:22.032372 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fe669d-e517-414e-a906-ddc6b5017b50 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:22.037218 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc62cc8-73b5-4222-95be-28dd7933a709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:22.037825 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc62cc8-73b5-4222-95be-28dd7933a709 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:22.038105 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc62cc8-73b5-4222-95be-28dd7933a709 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:22.038377 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc62cc8-73b5-4222-95be-28dd7933a709 admin admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:22.039017 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc62cc8-73b5-4222-95be-28dd7933a709 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:22.040939 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc62cc8-73b5-4222-95be-28dd7933a709 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:22.155186 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1019/2051] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 08:43:22 2022] POST /identity/v3/projects => generated 373 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:43:22.159169 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06471919-7c5c-402b-8278-fc749fcb160b admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:22.169762 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06471919-7c5c-402b-8278-fc749fcb160b admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:22.229441 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca06310-7cad-4653-98eb-92aabce5defd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:22.230054 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca06310-7cad-4653-98eb-92aabce5defd admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:22.230333 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca06310-7cad-4653-98eb-92aabce5defd admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:22.230633 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca06310-7cad-4653-98eb-92aabce5defd admin admin] PATH_INFO: `/v3/users` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:22.231267 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca06310-7cad-4653-98eb-92aabce5defd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:22.233201 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca06310-7cad-4653-98eb-92aabce5defd admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:22.338477 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1033/2052] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 08:43:22 2022] POST /identity/v3/users => generated 287 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:43:22.342509 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc62cc8-73b5-4222-95be-28dd7933a709 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:22.353218 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc62cc8-73b5-4222-95be-28dd7933a709 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:22.357969 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e135bfb4-86fe-478f-a835-78273c179af9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:22.360013 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e135bfb4-86fe-478f-a835-78273c179af9 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:22.360013 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e135bfb4-86fe-478f-a835-78273c179af9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:22.360013 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e135bfb4-86fe-478f-a835-78273c179af9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:22.360013 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e135bfb4-86fe-478f-a835-78273c179af9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:22.361472 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e135bfb4-86fe-478f-a835-78273c179af9 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:22.372653 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1020/2053] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:43:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:43:22.376696 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca06310-7cad-4653-98eb-92aabce5defd admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:22.387790 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca06310-7cad-4653-98eb-92aabce5defd admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:22.392546 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9545ac4e-796d-4ced-a816-3ad15331ba97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:22.393182 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9545ac4e-796d-4ced-a816-3ad15331ba97 admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:22.393459 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9545ac4e-796d-4ced-a816-3ad15331ba97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:22.393726 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9545ac4e-796d-4ced-a816-3ad15331ba97 admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:22.394393 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9545ac4e-796d-4ced-a816-3ad15331ba97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:22.396011 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9545ac4e-796d-4ced-a816-3ad15331ba97 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:22.407351 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1034/2054] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:43:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 08:43:22.411513 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e135bfb4-86fe-478f-a835-78273c179af9 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:22.422239 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e135bfb4-86fe-478f-a835-78273c179af9 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:22.426964 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb2a47a-4502-4518-b2f9-ac4fe444b221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:22.429124 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb2a47a-4502-4518-b2f9-ac4fe444b221 admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:22.429124 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb2a47a-4502-4518-b2f9-ac4fe444b221 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:22.429124 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb2a47a-4502-4518-b2f9-ac4fe444b221 admin admin] PATH_INFO: `/v3/projects/fa5c0f256e404409a5939c1d9e195b1c/users/28a3dda0c36a4b08aa2a314063005aa2/roles/44b64db484f24a46bf0b7ced46b519bf` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:22.444052 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb2a47a-4502-4518-b2f9-ac4fe444b221 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa5c0f256e404409a5939c1d9e195b1c, user_id=28a3dda0c36a4b08aa2a314063005aa2, role_id=44b64db484f24a46bf0b7ced46b519bf)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:22.446832 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb2a47a-4502-4518-b2f9-ac4fe444b221 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:22.526060 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1021/2055] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 08:43:22 2022] PUT /identity/v3/projects/fa5c0f256e404409a5939c1d9e195b1c/users/28a3dda0c36a4b08aa2a314063005aa2/roles/44b64db484f24a46bf0b7ced46b519bf => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:43:22.537295 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc54a5e-560b-4ca5-90b7-d4d6d0e32b87 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:22.537588 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc54a5e-560b-4ca5-90b7-d4d6d0e32b87 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:22.537997 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc54a5e-560b-4ca5-90b7-d4d6d0e32b87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:22.575647 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-4bc54a5e-560b-4ca5-90b7-d4d6d0e32b87 None None] MFA Rules not processed for user `28a3dda0c36a4b08aa2a314063005aa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:43:22.576847 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-4bc54a5e-560b-4ca5-90b7-d4d6d0e32b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:22.641613 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1035/2056] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:43:22 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:43:22.652103 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb2a47a-4502-4518-b2f9-ac4fe444b221 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:22.653810 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-2eb2a47a-4502-4518-b2f9-ac4fe444b221 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:22.711110 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb2a47a-4502-4518-b2f9-ac4fe444b221 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:22.714823 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757fbe6a-91d4-4c25-9623-bba784788358 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ba74e28ced4cbba857f54222dfbe41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:22.715432 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757fbe6a-91d4-4c25-9623-bba784788358 service cinder] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:22.715711 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757fbe6a-91d4-4c25-9623-bba784788358 service cinder] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:22.715981 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757fbe6a-91d4-4c25-9623-bba784788358 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:22.726049 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757fbe6a-91d4-4c25-9623-bba784788358 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:22.727506 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757fbe6a-91d4-4c25-9623-bba784788358 service cinder] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:22.736620 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1022/2057] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:43:22 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:43:23.583376 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc54a5e-560b-4ca5-90b7-d4d6d0e32b87 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:23.585496 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-4bc54a5e-560b-4ca5-90b7-d4d6d0e32b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:23.613387 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:43:23.655636 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc54a5e-560b-4ca5-90b7-d4d6d0e32b87 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:23.661718 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c5ea5c-dfe3-4435-b32e-c5aa7fbf459c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:23.662414 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c5ea5c-dfe3-4435-b32e-c5aa7fbf459c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:23.662697 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c5ea5c-dfe3-4435-b32e-c5aa7fbf459c admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:23.662965 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c5ea5c-dfe3-4435-b32e-c5aa7fbf459c admin admin] PATH_INFO: `/v3/users/94be600600904396895e9e7b981ca8d1` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:23.664155 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c5ea5c-dfe3-4435-b32e-c5aa7fbf459c admin admin] RBAC: Authorizing `identity:delete_user(user_id=94be600600904396895e9e7b981ca8d1)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:23.666153 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c5ea5c-dfe3-4435-b32e-c5aa7fbf459c admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:23.864307 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-e6c5ea5c-dfe3-4435-b32e-c5aa7fbf459c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94be600600904396895e9e7b981ca8d1'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:23.877422 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-e6c5ea5c-dfe3-4435-b32e-c5aa7fbf459c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94be600600904396895e9e7b981ca8d1'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:23.885539 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-e6c5ea5c-dfe3-4435-b32e-c5aa7fbf459c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94be600600904396895e9e7b981ca8d1'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:23.953231 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-e6c5ea5c-dfe3-4435-b32e-c5aa7fbf459c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94be600600904396895e9e7b981ca8d1'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:23.953871 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-e6c5ea5c-dfe3-4435-b32e-c5aa7fbf459c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94be600600904396895e9e7b981ca8d1'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:23.960448 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1036/2058] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 08:43:23 2022] DELETE /identity/v3/users/94be600600904396895e9e7b981ca8d1 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 08:43:23.965092 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79286b0-344f-4a7b-89ba-e2eb1f1ce0c6 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:23.965608 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79286b0-344f-4a7b-89ba-e2eb1f1ce0c6 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:23.965714 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79286b0-344f-4a7b-89ba-e2eb1f1ce0c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:23.985567 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-e79286b0-344f-4a7b-89ba-e2eb1f1ce0c6 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:43:23.986812 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-e79286b0-344f-4a7b-89ba-e2eb1f1ce0c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:24.047908 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1023/2059] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:43:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:43:24.058203 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c5ea5c-dfe3-4435-b32e-c5aa7fbf459c admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:24.059608 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-e6c5ea5c-dfe3-4435-b32e-c5aa7fbf459c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:24.115964 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c5ea5c-dfe3-4435-b32e-c5aa7fbf459c admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:24.119660 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9a1263-c4b0-4be3-9f31-dd9094526423 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:24.120264 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9a1263-c4b0-4be3-9f31-dd9094526423 service neutron] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:24.120546 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9a1263-c4b0-4be3-9f31-dd9094526423 service neutron] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:24.120821 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9a1263-c4b0-4be3-9f31-dd9094526423 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:24.130438 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9a1263-c4b0-4be3-9f31-dd9094526423 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:24.132608 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9a1263-c4b0-4be3-9f31-dd9094526423 service neutron] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:24.142136 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1037/2060] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:43:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:43:24.595454 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79286b0-344f-4a7b-89ba-e2eb1f1ce0c6 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:24.596868 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-e79286b0-344f-4a7b-89ba-e2eb1f1ce0c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:24.609535 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79286b0-344f-4a7b-89ba-e2eb1f1ce0c6 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:24.614919 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28937d59-ec4a-4597-939f-e7ba7e3f7183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:24.616137 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28937d59-ec4a-4597-939f-e7ba7e3f7183 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:24.616605 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28937d59-ec4a-4597-939f-e7ba7e3f7183 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:24.616903 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28937d59-ec4a-4597-939f-e7ba7e3f7183 admin admin] PATH_INFO: `/v3/projects/731bc09f5ded4e95a6a2b9d1fcf82c1c` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:24.618201 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28937d59-ec4a-4597-939f-e7ba7e3f7183 admin admin] RBAC: Authorizing `identity:delete_project(project_id=731bc09f5ded4e95a6a2b9d1fcf82c1c)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:24.621577 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28937d59-ec4a-4597-939f-e7ba7e3f7183 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:24.640745 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:43:24.730329 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-28937d59-ec4a-4597-939f-e7ba7e3f7183 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '731bc09f5ded4e95a6a2b9d1fcf82c1c', 'request_id': 'req-28937d59-ec4a-4597-939f-e7ba7e3f7183'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:24.799690 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-28937d59-ec4a-4597-939f-e7ba7e3f7183 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '731bc09f5ded4e95a6a2b9d1fcf82c1c', 'request_id': 'req-28937d59-ec4a-4597-939f-e7ba7e3f7183'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:24.806666 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-28937d59-ec4a-4597-939f-e7ba7e3f7183 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '731bc09f5ded4e95a6a2b9d1fcf82c1c', 'request_id': 'req-28937d59-ec4a-4597-939f-e7ba7e3f7183'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:24.819517 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-28937d59-ec4a-4597-939f-e7ba7e3f7183 admin admin] The token cache is being invalidate because project 731bc09f5ded4e95a6a2b9d1fcf82c1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 08:43:24.819798 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-28937d59-ec4a-4597-939f-e7ba7e3f7183 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:24.820570 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-28937d59-ec4a-4597-939f-e7ba7e3f7183 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:24.822174 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1024/2061] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 08:43:24 2022] DELETE /identity/v3/projects/731bc09f5ded4e95a6a2b9d1fcf82c1c => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:43:24.826958 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df92f66-f180-47e4-b4a9-237972129455 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:24.827436 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df92f66-f180-47e4-b4a9-237972129455 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:24.827527 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df92f66-f180-47e4-b4a9-237972129455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:24.847078 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-7df92f66-f180-47e4-b4a9-237972129455 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:43:24.848286 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-7df92f66-f180-47e4-b4a9-237972129455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:24.890418 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:43:24.909565 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1038/2062] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:43:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:43:24.918185 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2c6c83-1a29-421f-9192-69880f489e78 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:24.918470 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2c6c83-1a29-421f-9192-69880f489e78 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:24.918744 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2c6c83-1a29-421f-9192-69880f489e78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:24.939075 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-da2c6c83-1a29-421f-9192-69880f489e78 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:43:24.940312 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-da2c6c83-1a29-421f-9192-69880f489e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:24.949075 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1025/2063] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:43:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:43:24.953684 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df92f66-f180-47e4-b4a9-237972129455 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:24.963526 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df92f66-f180-47e4-b4a9-237972129455 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:24.968344 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ee211e-9e7c-4bad-ba02-1a01755d030d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:24.971573 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ee211e-9e7c-4bad-ba02-1a01755d030d admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:24.971573 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ee211e-9e7c-4bad-ba02-1a01755d030d admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:24.971573 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ee211e-9e7c-4bad-ba02-1a01755d030d admin admin] PATH_INFO: `/v3/domains` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:24.971573 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ee211e-9e7c-4bad-ba02-1a01755d030d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:43:24.971573 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ee211e-9e7c-4bad-ba02-1a01755d030d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:24.972364 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ee211e-9e7c-4bad-ba02-1a01755d030d admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:24.985189 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1039/2064] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Aug 10 08:43:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:43:24.989525 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2c6c83-1a29-421f-9192-69880f489e78 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:24.999527 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2c6c83-1a29-421f-9192-69880f489e78 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:25.005234 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eb180e-b3b5-4adf-9cfc-1befa575f2aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:25.006714 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eb180e-b3b5-4adf-9cfc-1befa575f2aa admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:25.007916 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eb180e-b3b5-4adf-9cfc-1befa575f2aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:25.007916 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eb180e-b3b5-4adf-9cfc-1befa575f2aa admin admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:25.007916 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45eb180e-b3b5-4adf-9cfc-1befa575f2aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:25.009966 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45eb180e-b3b5-4adf-9cfc-1befa575f2aa admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:25.098942 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1026/2065] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 08:43:24 2022] POST /identity/v3/projects => generated 369 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:43:25.104033 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ee211e-9e7c-4bad-ba02-1a01755d030d admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:25.115202 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ee211e-9e7c-4bad-ba02-1a01755d030d admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:25.178775 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5993bb-8763-4d86-a8d9-a4895200a343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:25.179601 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5993bb-8763-4d86-a8d9-a4895200a343 admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:25.179975 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5993bb-8763-4d86-a8d9-a4895200a343 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:25.180316 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5993bb-8763-4d86-a8d9-a4895200a343 admin admin] PATH_INFO: `/v3/users` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:25.181079 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5993bb-8763-4d86-a8d9-a4895200a343 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:25.183086 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5993bb-8763-4d86-a8d9-a4895200a343 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:25.260375 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1040/2066] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 08:43:25 2022] POST /identity/v3/users => generated 285 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:43:25.261701 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eb180e-b3b5-4adf-9cfc-1befa575f2aa admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:25.272448 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eb180e-b3b5-4adf-9cfc-1befa575f2aa admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:25.277338 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051897d9-32be-42f1-9e1f-f43ca0fb4d7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:25.277943 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051897d9-32be-42f1-9e1f-f43ca0fb4d7b admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:25.278241 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051897d9-32be-42f1-9e1f-f43ca0fb4d7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:25.278534 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051897d9-32be-42f1-9e1f-f43ca0fb4d7b admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:25.279203 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051897d9-32be-42f1-9e1f-f43ca0fb4d7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:25.280808 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051897d9-32be-42f1-9e1f-f43ca0fb4d7b admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:25.291446 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1027/2067] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:43:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 08:43:25.295408 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5993bb-8763-4d86-a8d9-a4895200a343 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:25.305805 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5993bb-8763-4d86-a8d9-a4895200a343 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:25.310799 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65da824c-ea53-42f1-802c-f69104560772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:25.311346 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65da824c-ea53-42f1-802c-f69104560772 admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:25.311648 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65da824c-ea53-42f1-802c-f69104560772 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:25.311921 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65da824c-ea53-42f1-802c-f69104560772 admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:25.312701 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65da824c-ea53-42f1-802c-f69104560772 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:25.314436 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65da824c-ea53-42f1-802c-f69104560772 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:25.330053 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1041/2068] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:43:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:43:25.331321 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051897d9-32be-42f1-9e1f-f43ca0fb4d7b admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:25.342232 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051897d9-32be-42f1-9e1f-f43ca0fb4d7b admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:25.347297 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1677a2-1929-4a34-aa87-ef22a172999f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:25.347952 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1677a2-1929-4a34-aa87-ef22a172999f admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:25.348240 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1677a2-1929-4a34-aa87-ef22a172999f admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:25.348513 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1677a2-1929-4a34-aa87-ef22a172999f admin admin] PATH_INFO: `/v3/projects/3cf74f95d149492ba14ed43020bf3f6e/users/144b9c05a1be449bb7aaf22857088237/roles/44b64db484f24a46bf0b7ced46b519bf` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:25.366215 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1677a2-1929-4a34-aa87-ef22a172999f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3cf74f95d149492ba14ed43020bf3f6e, user_id=144b9c05a1be449bb7aaf22857088237, role_id=44b64db484f24a46bf0b7ced46b519bf)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:25.372560 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1677a2-1929-4a34-aa87-ef22a172999f admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:25.465546 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1028/2069] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 08:43:25 2022] PUT /identity/v3/projects/3cf74f95d149492ba14ed43020bf3f6e/users/144b9c05a1be449bb7aaf22857088237/roles/44b64db484f24a46bf0b7ced46b519bf => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:43:25.477155 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6fb9f1-94c3-41c6-9b0f-c7e04362371f None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:25.477509 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6fb9f1-94c3-41c6-9b0f-c7e04362371f None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:25.477784 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6fb9f1-94c3-41c6-9b0f-c7e04362371f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:25.512619 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-9f6fb9f1-94c3-41c6-9b0f-c7e04362371f None None] MFA Rules not processed for user `144b9c05a1be449bb7aaf22857088237`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:43:25.514037 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-9f6fb9f1-94c3-41c6-9b0f-c7e04362371f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:25.596811 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1042/2070] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:43:25 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:43:25.610609 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1677a2-1929-4a34-aa87-ef22a172999f admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:25.612357 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-ac1677a2-1929-4a34-aa87-ef22a172999f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:25.634449 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9c70df-fd24-4095-8725-81027ca426ea None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:25.634655 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9c70df-fd24-4095-8725-81027ca426ea None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:25.634927 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9c70df-fd24-4095-8725-81027ca426ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:25.653676 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-bb9c70df-fd24-4095-8725-81027ca426ea None None] MFA Rules not processed for user `28a3dda0c36a4b08aa2a314063005aa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:43:25.654877 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-bb9c70df-fd24-4095-8725-81027ca426ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:25.688744 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:43:25.701726 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1677a2-1929-4a34-aa87-ef22a172999f admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:25.706127 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce4bbd3-16fd-4bcc-b8d7-dc99cd6b44bc service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3263af99fc2f48e79fd1e5b8a1e555a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:25.706946 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce4bbd3-16fd-4bcc-b8d7-dc99cd6b44bc service glance] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:25.707046 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce4bbd3-16fd-4bcc-b8d7-dc99cd6b44bc service glance] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:25.707303 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce4bbd3-16fd-4bcc-b8d7-dc99cd6b44bc service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:25.716790 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce4bbd3-16fd-4bcc-b8d7-dc99cd6b44bc service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:25.716968 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1043/2071] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:43:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:43:25.718046 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce4bbd3-16fd-4bcc-b8d7-dc99cd6b44bc service glance] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:25.733026 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1029/2072] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:43:25 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:43:33.948047 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b465093-cf53-411f-811b-c741f7ce1300 None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:33.948650 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b465093-cf53-411f-811b-c741f7ce1300 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:33.948650 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b465093-cf53-411f-811b-c741f7ce1300 None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:33.949979 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1044/2073] 10.222.0.57 () {54 vars in 921 bytes} [Wed Aug 10 08:43:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:43:33.959332 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaab5ca-dd09-46b6-a304-e6f5137d0da5 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:33.959795 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaab5ca-dd09-46b6-a304-e6f5137d0da5 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:33.959952 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaab5ca-dd09-46b6-a304-e6f5137d0da5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:33.979991 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-cbaab5ca-dd09-46b6-a304-e6f5137d0da5 None None] MFA Rules not processed for user `2912307c56594125b157f1cfdf7c96a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:43:33.981217 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-cbaab5ca-dd09-46b6-a304-e6f5137d0da5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:34.004112 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:43:34.058701 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1030/2074] 10.222.0.57 () {58 vars in 1018 bytes} [Wed Aug 10 08:43:33 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:43:34.088809 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b465093-cf53-411f-811b-c741f7ce1300 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:34.090583 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-1b465093-cf53-411f-811b-c741f7ce1300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:34.116962 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b465093-cf53-411f-811b-c741f7ce1300 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:34.123194 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cf1b8c-6411-4caa-b7cf-d11222c4cec4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2912307c56594125b157f1cfdf7c96a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:34.123790 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cf1b8c-6411-4caa-b7cf-d11222c4cec4 service placement] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:34.124137 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cf1b8c-6411-4caa-b7cf-d11222c4cec4 service placement] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:34.124477 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cf1b8c-6411-4caa-b7cf-d11222c4cec4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:34.134375 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cf1b8c-6411-4caa-b7cf-d11222c4cec4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:34.136007 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cf1b8c-6411-4caa-b7cf-d11222c4cec4 service placement] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:34.153758 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1045/2075] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:43:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:43:34.170978 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaab5ca-dd09-46b6-a304-e6f5137d0da5 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:34.173937 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-cbaab5ca-dd09-46b6-a304-e6f5137d0da5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:34.261464 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaab5ca-dd09-46b6-a304-e6f5137d0da5 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:34.266475 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881baa8a-5947-42fb-8d90-9c155e39f456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:34.267110 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881baa8a-5947-42fb-8d90-9c155e39f456 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:34.267389 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881baa8a-5947-42fb-8d90-9c155e39f456 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:34.267661 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881baa8a-5947-42fb-8d90-9c155e39f456 admin admin] PATH_INFO: `/v3/users/b82a623527ad4280a353270fc3d51ba1` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:34.268823 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881baa8a-5947-42fb-8d90-9c155e39f456 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b82a623527ad4280a353270fc3d51ba1)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:34.270750 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881baa8a-5947-42fb-8d90-9c155e39f456 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:34.464854 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-881baa8a-5947-42fb-8d90-9c155e39f456 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b82a623527ad4280a353270fc3d51ba1'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:34.480727 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-881baa8a-5947-42fb-8d90-9c155e39f456 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b82a623527ad4280a353270fc3d51ba1'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:34.488965 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-881baa8a-5947-42fb-8d90-9c155e39f456 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b82a623527ad4280a353270fc3d51ba1'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:34.488965 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-881baa8a-5947-42fb-8d90-9c155e39f456 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b82a623527ad4280a353270fc3d51ba1'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:34.489530 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-881baa8a-5947-42fb-8d90-9c155e39f456 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b82a623527ad4280a353270fc3d51ba1'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:34.570017 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1031/2076] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 08:43:34 2022] DELETE /identity/v3/users/b82a623527ad4280a353270fc3d51ba1 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:43:34.835076 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cf1b8c-6411-4caa-b7cf-d11222c4cec4 service placement] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:34.852286 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cf1b8c-6411-4caa-b7cf-d11222c4cec4 service placement] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:34.908078 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bade1eb-4d96-4905-b63b-bb15c4b1027c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:34.908699 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bade1eb-4d96-4905-b63b-bb15c4b1027c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:34.909005 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bade1eb-4d96-4905-b63b-bb15c4b1027c admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:34.909277 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bade1eb-4d96-4905-b63b-bb15c4b1027c admin admin] PATH_INFO: `/v3/projects/1cdf045fd98443afa186f8d3e2bf690e` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:34.910382 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bade1eb-4d96-4905-b63b-bb15c4b1027c admin admin] RBAC: Authorizing `identity:delete_project(project_id=1cdf045fd98443afa186f8d3e2bf690e)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:34.912310 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bade1eb-4d96-4905-b63b-bb15c4b1027c admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:35.006231 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-2bade1eb-4d96-4905-b63b-bb15c4b1027c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1cdf045fd98443afa186f8d3e2bf690e', 'request_id': 'req-2bade1eb-4d96-4905-b63b-bb15c4b1027c'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:35.011702 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-2bade1eb-4d96-4905-b63b-bb15c4b1027c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1cdf045fd98443afa186f8d3e2bf690e', 'request_id': 'req-2bade1eb-4d96-4905-b63b-bb15c4b1027c'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:35.073530 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-2bade1eb-4d96-4905-b63b-bb15c4b1027c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1cdf045fd98443afa186f8d3e2bf690e', 'request_id': 'req-2bade1eb-4d96-4905-b63b-bb15c4b1027c'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:35.085936 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-2bade1eb-4d96-4905-b63b-bb15c4b1027c admin admin] The token cache is being invalidate because project 1cdf045fd98443afa186f8d3e2bf690e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 08:43:35.086713 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-2bade1eb-4d96-4905-b63b-bb15c4b1027c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:35.087403 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-2bade1eb-4d96-4905-b63b-bb15c4b1027c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:35.088909 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1046/2077] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 08:43:34 2022] DELETE /identity/v3/projects/1cdf045fd98443afa186f8d3e2bf690e => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:43:35.093877 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a08dbab-98b9-4fbb-9c01-08728009e26b None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:35.094336 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a08dbab-98b9-4fbb-9c01-08728009e26b None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:35.094448 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a08dbab-98b9-4fbb-9c01-08728009e26b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:35.117235 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-1a08dbab-98b9-4fbb-9c01-08728009e26b None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:43:35.118726 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-1a08dbab-98b9-4fbb-9c01-08728009e26b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:35.153041 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:43:35.182410 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1032/2078] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:43:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:43:35.190191 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fd7428-9551-4c5e-9b24-4c7095de49e8 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:35.190472 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fd7428-9551-4c5e-9b24-4c7095de49e8 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:35.190742 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fd7428-9551-4c5e-9b24-4c7095de49e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:35.210232 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-b6fd7428-9551-4c5e-9b24-4c7095de49e8 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:43:35.212202 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-b6fd7428-9551-4c5e-9b24-4c7095de49e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:35.219854 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1047/2079] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:43:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:43:35.223980 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a08dbab-98b9-4fbb-9c01-08728009e26b None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:35.235111 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a08dbab-98b9-4fbb-9c01-08728009e26b None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:35.239641 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a733de-d940-4a2e-bece-99af735d35c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:35.240230 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a733de-d940-4a2e-bece-99af735d35c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:35.240505 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a733de-d940-4a2e-bece-99af735d35c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:35.240795 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a733de-d940-4a2e-bece-99af735d35c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:35.241468 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a733de-d940-4a2e-bece-99af735d35c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:43:35.241765 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a733de-d940-4a2e-bece-99af735d35c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:35.243387 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a733de-d940-4a2e-bece-99af735d35c5 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:35.254986 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1033/2080] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Aug 10 08:43:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:43:35.259822 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fd7428-9551-4c5e-9b24-4c7095de49e8 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:35.268908 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fd7428-9551-4c5e-9b24-4c7095de49e8 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:35.273560 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3f033f-ba52-4702-9882-4d787c78287f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:35.274661 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3f033f-ba52-4702-9882-4d787c78287f admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:35.274931 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3f033f-ba52-4702-9882-4d787c78287f admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:35.275203 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3f033f-ba52-4702-9882-4d787c78287f admin admin] PATH_INFO: `/v3/projects` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:35.275814 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3f033f-ba52-4702-9882-4d787c78287f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:35.277792 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3f033f-ba52-4702-9882-4d787c78287f admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:35.365006 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1048/2081] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 08:43:35 2022] POST /identity/v3/projects => generated 373 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:43:35.366585 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a733de-d940-4a2e-bece-99af735d35c5 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:35.377616 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a733de-d940-4a2e-bece-99af735d35c5 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:35.441045 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724f2d2f-d493-4255-9cdc-d322163b96e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:35.441813 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724f2d2f-d493-4255-9cdc-d322163b96e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:35.442103 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724f2d2f-d493-4255-9cdc-d322163b96e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:35.442394 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724f2d2f-d493-4255-9cdc-d322163b96e1 admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:35.449565 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724f2d2f-d493-4255-9cdc-d322163b96e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:35.451545 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724f2d2f-d493-4255-9cdc-d322163b96e1 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:35.560563 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1034/2082] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 08:43:35 2022] POST /identity/v3/users => generated 287 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:43:35.569464 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3f033f-ba52-4702-9882-4d787c78287f admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:35.577461 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3f033f-ba52-4702-9882-4d787c78287f admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:35.582493 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434395cb-e42a-40ff-be78-5047d7797848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:35.583221 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434395cb-e42a-40ff-be78-5047d7797848 admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:35.583568 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434395cb-e42a-40ff-be78-5047d7797848 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:35.583921 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434395cb-e42a-40ff-be78-5047d7797848 admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:35.584713 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434395cb-e42a-40ff-be78-5047d7797848 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:35.587202 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434395cb-e42a-40ff-be78-5047d7797848 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:35.597684 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1049/2083] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:43:35 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 08:43:35.601723 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724f2d2f-d493-4255-9cdc-d322163b96e1 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:35.612069 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724f2d2f-d493-4255-9cdc-d322163b96e1 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:35.616812 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa2bc04-1b49-4b14-baea-b1024bd937c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:35.617493 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa2bc04-1b49-4b14-baea-b1024bd937c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:35.617773 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa2bc04-1b49-4b14-baea-b1024bd937c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:35.618123 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa2bc04-1b49-4b14-baea-b1024bd937c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:35.618770 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa2bc04-1b49-4b14-baea-b1024bd937c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:35.620406 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa2bc04-1b49-4b14-baea-b1024bd937c9 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:35.631596 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1035/2084] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:43:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:43:35.636497 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434395cb-e42a-40ff-be78-5047d7797848 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:35.646898 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434395cb-e42a-40ff-be78-5047d7797848 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:35.655536 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbc0117-9638-4b43-bc81-65e515c549df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:35.656969 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbc0117-9638-4b43-bc81-65e515c549df admin admin] REQUEST_METHOD: `PUT` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:35.657362 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbc0117-9638-4b43-bc81-65e515c549df admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:35.657745 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbc0117-9638-4b43-bc81-65e515c549df admin admin] PATH_INFO: `/v3/projects/39553e244fc94eccb50d56414183f953/users/44d3d8dec4394cad81f18cdfdd020b09/roles/44b64db484f24a46bf0b7ced46b519bf` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:35.674227 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbc0117-9638-4b43-bc81-65e515c549df admin admin] RBAC: Authorizing `identity:create_grant(project_id=39553e244fc94eccb50d56414183f953, user_id=44d3d8dec4394cad81f18cdfdd020b09, role_id=44b64db484f24a46bf0b7ced46b519bf)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:35.677047 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbc0117-9638-4b43-bc81-65e515c549df admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:35.737465 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:43:35.743994 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1050/2085] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 08:43:35 2022] PUT /identity/v3/projects/39553e244fc94eccb50d56414183f953/users/44d3d8dec4394cad81f18cdfdd020b09/roles/44b64db484f24a46bf0b7ced46b519bf => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:43:35.755072 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7932f15-e342-4f63-9e09-97d6ef3153f7 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:35.755362 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7932f15-e342-4f63-9e09-97d6ef3153f7 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:35.755861 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7932f15-e342-4f63-9e09-97d6ef3153f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:35.786104 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-d7932f15-e342-4f63-9e09-97d6ef3153f7 None None] MFA Rules not processed for user `44d3d8dec4394cad81f18cdfdd020b09`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:43:35.787314 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-d7932f15-e342-4f63-9e09-97d6ef3153f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:35.850949 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1036/2086] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:43:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 08:43:35.862202 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbc0117-9638-4b43-bc81-65e515c549df admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:35.864017 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-ffbc0117-9638-4b43-bc81-65e515c549df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:35.945383 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbc0117-9638-4b43-bc81-65e515c549df admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:35.951073 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13554f2-4404-48ab-8871-e651d0a5efa7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a943008193914f76a663d489bd73e873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:35.951715 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13554f2-4404-48ab-8871-e651d0a5efa7 service nova] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:35.952000 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13554f2-4404-48ab-8871-e651d0a5efa7 service nova] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:35.952274 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13554f2-4404-48ab-8871-e651d0a5efa7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:35.959923 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13554f2-4404-48ab-8871-e651d0a5efa7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:35.962638 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13554f2-4404-48ab-8871-e651d0a5efa7 service nova] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:35.971338 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1051/2087] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:43:35 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:43:40.309400 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7932f15-e342-4f63-9e09-97d6ef3153f7 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:40.366335 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:43:40.397575 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7932f15-e342-4f63-9e09-97d6ef3153f7 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:40.407077 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5300fee-e20a-4fc6-906e-f5c87c7454f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:40.407880 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5300fee-e20a-4fc6-906e-f5c87c7454f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:40.408177 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5300fee-e20a-4fc6-906e-f5c87c7454f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:40.408513 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5300fee-e20a-4fc6-906e-f5c87c7454f2 admin admin] PATH_INFO: `/v3/users/44d3d8dec4394cad81f18cdfdd020b09` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:40.409758 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5300fee-e20a-4fc6-906e-f5c87c7454f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=44d3d8dec4394cad81f18cdfdd020b09)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:40.411954 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5300fee-e20a-4fc6-906e-f5c87c7454f2 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:40.601166 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-b5300fee-e20a-4fc6-906e-f5c87c7454f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44d3d8dec4394cad81f18cdfdd020b09'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:40.614350 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-b5300fee-e20a-4fc6-906e-f5c87c7454f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44d3d8dec4394cad81f18cdfdd020b09'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:40.621453 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-b5300fee-e20a-4fc6-906e-f5c87c7454f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44d3d8dec4394cad81f18cdfdd020b09'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:40.622392 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-b5300fee-e20a-4fc6-906e-f5c87c7454f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44d3d8dec4394cad81f18cdfdd020b09'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:40.623001 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-b5300fee-e20a-4fc6-906e-f5c87c7454f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44d3d8dec4394cad81f18cdfdd020b09'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:40.707174 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1037/2088] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 08:43:40 2022] DELETE /identity/v3/users/44d3d8dec4394cad81f18cdfdd020b09 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:43:40.711790 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcb8a42-ca8a-4039-9386-c6aa26347459 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:40.712377 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcb8a42-ca8a-4039-9386-c6aa26347459 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:40.712500 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcb8a42-ca8a-4039-9386-c6aa26347459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:40.732606 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-1dcb8a42-ca8a-4039-9386-c6aa26347459 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:43:40.733982 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-1dcb8a42-ca8a-4039-9386-c6aa26347459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:40.794440 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1052/2089] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:43:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:43:40.804269 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5300fee-e20a-4fc6-906e-f5c87c7454f2 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:40.810413 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-b5300fee-e20a-4fc6-906e-f5c87c7454f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:40.862178 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5300fee-e20a-4fc6-906e-f5c87c7454f2 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:40.865759 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d9e43d-d8d2-4fa4-bd1b-f744395c46e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:40.866353 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d9e43d-d8d2-4fa4-bd1b-f744395c46e0 service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:40.866624 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d9e43d-d8d2-4fa4-bd1b-f744395c46e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:40.866886 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d9e43d-d8d2-4fa4-bd1b-f744395c46e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:40.875836 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d9e43d-d8d2-4fa4-bd1b-f744395c46e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:40.877282 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d9e43d-d8d2-4fa4-bd1b-f744395c46e0 service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:40.887030 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1038/2090] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:43:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:43:41.282642 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcb8a42-ca8a-4039-9386-c6aa26347459 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:41.282642 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-1dcb8a42-ca8a-4039-9386-c6aa26347459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:41.298910 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcb8a42-ca8a-4039-9386-c6aa26347459 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:41.303973 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb22bd8-0a63-4d91-a3da-63f3ff31e6bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:41.304594 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb22bd8-0a63-4d91-a3da-63f3ff31e6bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:41.304872 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb22bd8-0a63-4d91-a3da-63f3ff31e6bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:41.305203 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb22bd8-0a63-4d91-a3da-63f3ff31e6bc admin admin] PATH_INFO: `/v3/projects/39553e244fc94eccb50d56414183f953` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:41.306512 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb22bd8-0a63-4d91-a3da-63f3ff31e6bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=39553e244fc94eccb50d56414183f953)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:41.311947 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb22bd8-0a63-4d91-a3da-63f3ff31e6bc admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:41.428788 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-1eb22bd8-0a63-4d91-a3da-63f3ff31e6bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39553e244fc94eccb50d56414183f953', 'request_id': 'req-1eb22bd8-0a63-4d91-a3da-63f3ff31e6bc'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:41.435312 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-1eb22bd8-0a63-4d91-a3da-63f3ff31e6bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39553e244fc94eccb50d56414183f953', 'request_id': 'req-1eb22bd8-0a63-4d91-a3da-63f3ff31e6bc'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:41.490558 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-1eb22bd8-0a63-4d91-a3da-63f3ff31e6bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39553e244fc94eccb50d56414183f953', 'request_id': 'req-1eb22bd8-0a63-4d91-a3da-63f3ff31e6bc'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:41.505156 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-1eb22bd8-0a63-4d91-a3da-63f3ff31e6bc admin admin] The token cache is being invalidate because project 39553e244fc94eccb50d56414183f953 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 08:43:41.505520 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-1eb22bd8-0a63-4d91-a3da-63f3ff31e6bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:41.506839 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-1eb22bd8-0a63-4d91-a3da-63f3ff31e6bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:41.508684 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1053/2091] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 08:43:41 2022] DELETE /identity/v3/projects/39553e244fc94eccb50d56414183f953 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:43:41.514085 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8b62d7-d36b-4c0a-be7b-84fc97e0667b None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:41.515164 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8b62d7-d36b-4c0a-be7b-84fc97e0667b None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:41.515743 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8b62d7-d36b-4c0a-be7b-84fc97e0667b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:41.538948 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-aa8b62d7-d36b-4c0a-be7b-84fc97e0667b None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:43:41.540295 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-aa8b62d7-d36b-4c0a-be7b-84fc97e0667b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:41.595158 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:43:41.604636 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1039/2092] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:43:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:43:41.613869 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cef3ce1-cef3-4c9c-a4e1-07e88327e265 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:41.614236 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cef3ce1-cef3-4c9c-a4e1-07e88327e265 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:41.614715 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cef3ce1-cef3-4c9c-a4e1-07e88327e265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:41.631175 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:43:41.638346 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-7cef3ce1-cef3-4c9c-a4e1-07e88327e265 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:43:41.639600 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-7cef3ce1-cef3-4c9c-a4e1-07e88327e265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:41.647547 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1054/2093] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:43:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:43:41.652095 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8b62d7-d36b-4c0a-be7b-84fc97e0667b None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:41.661764 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8b62d7-d36b-4c0a-be7b-84fc97e0667b None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:41.667893 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd13c05-0ba9-4cda-8745-925dc2766709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:41.668497 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd13c05-0ba9-4cda-8745-925dc2766709 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:41.671347 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd13c05-0ba9-4cda-8745-925dc2766709 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:41.671347 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd13c05-0ba9-4cda-8745-925dc2766709 admin admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:41.671347 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd13c05-0ba9-4cda-8745-925dc2766709 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:43:41.671347 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd13c05-0ba9-4cda-8745-925dc2766709 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:41.671803 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd13c05-0ba9-4cda-8745-925dc2766709 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:41.684198 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1040/2094] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Aug 10 08:43:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:43:41.689229 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cef3ce1-cef3-4c9c-a4e1-07e88327e265 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:41.699125 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cef3ce1-cef3-4c9c-a4e1-07e88327e265 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:41.704003 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f06b1a1-034b-4945-a876-fc5153d2ab5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:41.704608 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f06b1a1-034b-4945-a876-fc5153d2ab5c admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:41.704900 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f06b1a1-034b-4945-a876-fc5153d2ab5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:41.705251 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f06b1a1-034b-4945-a876-fc5153d2ab5c admin admin] PATH_INFO: `/v3/projects` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:41.705928 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f06b1a1-034b-4945-a876-fc5153d2ab5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:41.707831 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f06b1a1-034b-4945-a876-fc5153d2ab5c admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:41.816099 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1055/2095] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 08:43:41 2022] POST /identity/v3/projects => generated 405 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 08:43:41.817990 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd13c05-0ba9-4cda-8745-925dc2766709 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:41.828095 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd13c05-0ba9-4cda-8745-925dc2766709 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:41.890436 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e76a72-01c0-4e35-b264-65ddc6b2b498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:41.891487 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e76a72-01c0-4e35-b264-65ddc6b2b498 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:41.892313 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e76a72-01c0-4e35-b264-65ddc6b2b498 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:41.893041 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e76a72-01c0-4e35-b264-65ddc6b2b498 admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:41.893831 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e76a72-01c0-4e35-b264-65ddc6b2b498 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:41.896160 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e76a72-01c0-4e35-b264-65ddc6b2b498 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:41.965729 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1041/2096] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 08:43:41 2022] POST /identity/v3/users => generated 303 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:43:41.970012 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f06b1a1-034b-4945-a876-fc5153d2ab5c admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:41.980777 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f06b1a1-034b-4945-a876-fc5153d2ab5c admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:41.985291 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a030008-48b2-48e1-a0f0-83c7193f2f8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:41.988532 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a030008-48b2-48e1-a0f0-83c7193f2f8a admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:41.988532 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a030008-48b2-48e1-a0f0-83c7193f2f8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:41.988532 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a030008-48b2-48e1-a0f0-83c7193f2f8a admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:41.988532 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a030008-48b2-48e1-a0f0-83c7193f2f8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:41.988888 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a030008-48b2-48e1-a0f0-83c7193f2f8a admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:41.998897 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1056/2097] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:43:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:43:42.002584 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e76a72-01c0-4e35-b264-65ddc6b2b498 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:42.013689 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e76a72-01c0-4e35-b264-65ddc6b2b498 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:42.018564 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4d2548-8234-4fed-8e80-ab01370fd3c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:42.019150 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4d2548-8234-4fed-8e80-ab01370fd3c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:42.019425 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4d2548-8234-4fed-8e80-ab01370fd3c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:42.019694 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4d2548-8234-4fed-8e80-ab01370fd3c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:42.020412 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4d2548-8234-4fed-8e80-ab01370fd3c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:42.022044 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4d2548-8234-4fed-8e80-ab01370fd3c6 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:42.034560 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1042/2098] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:43:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:43:42.038704 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a030008-48b2-48e1-a0f0-83c7193f2f8a admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:42.049736 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a030008-48b2-48e1-a0f0-83c7193f2f8a admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:42.054229 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fd96ca-b324-4232-ac28-3298a587cbe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:42.054873 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fd96ca-b324-4232-ac28-3298a587cbe1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:42.055149 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fd96ca-b324-4232-ac28-3298a587cbe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:42.055564 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fd96ca-b324-4232-ac28-3298a587cbe1 admin admin] PATH_INFO: `/v3/projects/47d891f8ea0c455b8879400164429ba8/users/be27155bb40e4d28913714e2e5670d04/roles/44b64db484f24a46bf0b7ced46b519bf` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:42.070581 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fd96ca-b324-4232-ac28-3298a587cbe1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=47d891f8ea0c455b8879400164429ba8, user_id=be27155bb40e4d28913714e2e5670d04, role_id=44b64db484f24a46bf0b7ced46b519bf)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:42.074732 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fd96ca-b324-4232-ac28-3298a587cbe1 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:42.153768 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1057/2099] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 08:43:42 2022] PUT /identity/v3/projects/47d891f8ea0c455b8879400164429ba8/users/be27155bb40e4d28913714e2e5670d04/roles/44b64db484f24a46bf0b7ced46b519bf => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:43:42.158968 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751eb6b0-5502-42ab-abb9-002201d22e34 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:42.159247 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751eb6b0-5502-42ab-abb9-002201d22e34 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:42.159511 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751eb6b0-5502-42ab-abb9-002201d22e34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:42.179111 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-751eb6b0-5502-42ab-abb9-002201d22e34 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:43:42.180340 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-751eb6b0-5502-42ab-abb9-002201d22e34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:42.238290 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1043/2100] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:43:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:43:42.248074 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fd96ca-b324-4232-ac28-3298a587cbe1 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:42.249769 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-96fd96ca-b324-4232-ac28-3298a587cbe1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:42.306658 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fd96ca-b324-4232-ac28-3298a587cbe1 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:42.311000 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654af561-922d-41f3-9379-9ee34addc3df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:42.311634 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654af561-922d-41f3-9379-9ee34addc3df service neutron] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:42.311911 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654af561-922d-41f3-9379-9ee34addc3df service neutron] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:42.312177 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654af561-922d-41f3-9379-9ee34addc3df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:42.320940 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654af561-922d-41f3-9379-9ee34addc3df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:42.322967 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654af561-922d-41f3-9379-9ee34addc3df service neutron] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:42.332470 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1058/2101] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:43:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:43:43.474068 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce307a3-5202-4971-b195-64ecc17f546d None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:43.474573 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce307a3-5202-4971-b195-64ecc17f546d None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:43.474791 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce307a3-5202-4971-b195-64ecc17f546d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:43.506573 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-8ce307a3-5202-4971-b195-64ecc17f546d None None] MFA Rules not processed for user `be27155bb40e4d28913714e2e5670d04`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:43:43.507843 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-8ce307a3-5202-4971-b195-64ecc17f546d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:43.566971 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:43:43.580643 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1044/2102] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:43:43 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:43:43.585041 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654af561-922d-41f3-9379-9ee34addc3df service neutron] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:43.595688 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654af561-922d-41f3-9379-9ee34addc3df service neutron] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:43.600902 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e91303-f37f-4589-afd8-00d82e71154c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:43.601536 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e91303-f37f-4589-afd8-00d82e71154c admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:43.601821 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e91303-f37f-4589-afd8-00d82e71154c admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:43.602092 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e91303-f37f-4589-afd8-00d82e71154c admin admin] PATH_INFO: `/v3/projects` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:43.602765 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e91303-f37f-4589-afd8-00d82e71154c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:43.604894 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e91303-f37f-4589-afd8-00d82e71154c admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:43.685763 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1059/2103] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 08:43:43 2022] POST /identity/v3/projects => generated 403 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 08:43:43.689963 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce307a3-5202-4971-b195-64ecc17f546d None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:43.701571 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce307a3-5202-4971-b195-64ecc17f546d None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:43.758002 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed3a0e2-b215-4c36-a229-93223a36e493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:43.758604 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed3a0e2-b215-4c36-a229-93223a36e493 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:43.758889 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed3a0e2-b215-4c36-a229-93223a36e493 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:43.759158 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed3a0e2-b215-4c36-a229-93223a36e493 admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:43.759775 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed3a0e2-b215-4c36-a229-93223a36e493 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:43.762257 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed3a0e2-b215-4c36-a229-93223a36e493 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:43.867394 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1045/2104] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 08:43:43 2022] POST /identity/v3/users => generated 308 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 08:43:43.871595 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e91303-f37f-4589-afd8-00d82e71154c admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:43.882757 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e91303-f37f-4589-afd8-00d82e71154c admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:43.888574 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c39567-7bff-4f19-a827-4512494a62c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:43.889272 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c39567-7bff-4f19-a827-4512494a62c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:43.889556 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c39567-7bff-4f19-a827-4512494a62c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:43.889825 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c39567-7bff-4f19-a827-4512494a62c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:43.890580 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c39567-7bff-4f19-a827-4512494a62c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:43.892161 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c39567-7bff-4f19-a827-4512494a62c5 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:43.904382 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1060/2105] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:43:43 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:43:43.908586 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed3a0e2-b215-4c36-a229-93223a36e493 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:43.919307 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed3a0e2-b215-4c36-a229-93223a36e493 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:43.923903 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea4dc3e-9370-4c99-979d-75705d5adcb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:43.924547 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea4dc3e-9370-4c99-979d-75705d5adcb8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:43.924823 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea4dc3e-9370-4c99-979d-75705d5adcb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:43.925185 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea4dc3e-9370-4c99-979d-75705d5adcb8 admin admin] PATH_INFO: `/v3/projects/dc73e10e90104bac9e3acfb07efcfe83/users/acde448c012a4c6793379092795cb28f/roles/ec3751692d9c4ecf81b4a71c936c46be` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:43.941239 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea4dc3e-9370-4c99-979d-75705d5adcb8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc73e10e90104bac9e3acfb07efcfe83, user_id=acde448c012a4c6793379092795cb28f, role_id=ec3751692d9c4ecf81b4a71c936c46be)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:43.943861 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea4dc3e-9370-4c99-979d-75705d5adcb8 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:44.033698 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1046/2106] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 08:43:43 2022] PUT /identity/v3/projects/dc73e10e90104bac9e3acfb07efcfe83/users/acde448c012a4c6793379092795cb28f/roles/ec3751692d9c4ecf81b4a71c936c46be => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 08:43:44.041228 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c39567-7bff-4f19-a827-4512494a62c5 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:44.051312 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c39567-7bff-4f19-a827-4512494a62c5 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:44.106146 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c7c877-d1ac-4917-9cfc-214ea0621015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:44.106740 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c7c877-d1ac-4917-9cfc-214ea0621015 admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:44.107015 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c7c877-d1ac-4917-9cfc-214ea0621015 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:44.107282 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c7c877-d1ac-4917-9cfc-214ea0621015 admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:44.107905 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c7c877-d1ac-4917-9cfc-214ea0621015 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:44.109509 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c7c877-d1ac-4917-9cfc-214ea0621015 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:44.118906 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1061/2107] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:43:44 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:43:44.124193 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea4dc3e-9370-4c99-979d-75705d5adcb8 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:44.134902 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea4dc3e-9370-4c99-979d-75705d5adcb8 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:44.139668 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5c27d2-f338-4549-93f7-e1d78d471e50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:44.140310 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5c27d2-f338-4549-93f7-e1d78d471e50 admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:44.140592 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5c27d2-f338-4549-93f7-e1d78d471e50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:44.140869 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5c27d2-f338-4549-93f7-e1d78d471e50 admin admin] PATH_INFO: `/v3/projects/dc73e10e90104bac9e3acfb07efcfe83/users/acde448c012a4c6793379092795cb28f/roles/ec3751692d9c4ecf81b4a71c936c46be` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:44.142792 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5c27d2-f338-4549-93f7-e1d78d471e50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc73e10e90104bac9e3acfb07efcfe83, user_id=acde448c012a4c6793379092795cb28f, role_id=ec3751692d9c4ecf81b4a71c936c46be)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:44.145757 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5c27d2-f338-4549-93f7-e1d78d471e50 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:44.157116 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1047/2108] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 08:43:44 2022] PUT /identity/v3/projects/dc73e10e90104bac9e3acfb07efcfe83/users/acde448c012a4c6793379092795cb28f/roles/ec3751692d9c4ecf81b4a71c936c46be => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:43:45.511941 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8658c5b-259b-40da-8254-99e7b1942fa3 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:45.512417 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8658c5b-259b-40da-8254-99e7b1942fa3 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:45.512524 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8658c5b-259b-40da-8254-99e7b1942fa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:45.534049 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:43:45.544767 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-f8658c5b-259b-40da-8254-99e7b1942fa3 None None] MFA Rules not processed for user `acde448c012a4c6793379092795cb28f`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:43:45.546057 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-f8658c5b-259b-40da-8254-99e7b1942fa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:45.613539 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1062/2109] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:43:45 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:43:47.246593 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5c27d2-f338-4549-93f7-e1d78d471e50 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:47.313692 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5c27d2-f338-4549-93f7-e1d78d471e50 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:47.318242 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b3733c-04be-445e-b017-34bd49aafb68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:47.318884 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b3733c-04be-445e-b017-34bd49aafb68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:47.319162 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b3733c-04be-445e-b017-34bd49aafb68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:47.319433 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b3733c-04be-445e-b017-34bd49aafb68 admin admin] PATH_INFO: `/v3/users/be27155bb40e4d28913714e2e5670d04` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:47.320558 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b3733c-04be-445e-b017-34bd49aafb68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be27155bb40e4d28913714e2e5670d04)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:47.322516 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b3733c-04be-445e-b017-34bd49aafb68 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:47.441166 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-33b3733c-04be-445e-b017-34bd49aafb68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be27155bb40e4d28913714e2e5670d04'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:47.442294 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:43:47.454136 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-33b3733c-04be-445e-b017-34bd49aafb68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be27155bb40e4d28913714e2e5670d04'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:47.461270 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-33b3733c-04be-445e-b017-34bd49aafb68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be27155bb40e4d28913714e2e5670d04'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:47.461978 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-33b3733c-04be-445e-b017-34bd49aafb68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be27155bb40e4d28913714e2e5670d04'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:47.462575 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-33b3733c-04be-445e-b017-34bd49aafb68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be27155bb40e4d28913714e2e5670d04'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:47.542425 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1048/2110] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 08:43:47 2022] DELETE /identity/v3/users/be27155bb40e4d28913714e2e5670d04 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:43:47.774358 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8658c5b-259b-40da-8254-99e7b1942fa3 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:47.803230 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8658c5b-259b-40da-8254-99e7b1942fa3 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:47.890118 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2841399-471e-46ed-8073-e4f9427523dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:47.890825 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2841399-471e-46ed-8073-e4f9427523dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:47.891172 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2841399-471e-46ed-8073-e4f9427523dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:47.891513 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2841399-471e-46ed-8073-e4f9427523dd admin admin] PATH_INFO: `/v3/projects/47d891f8ea0c455b8879400164429ba8` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:47.892700 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2841399-471e-46ed-8073-e4f9427523dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=47d891f8ea0c455b8879400164429ba8)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:47.894759 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2841399-471e-46ed-8073-e4f9427523dd admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:48.063538 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-a2841399-471e-46ed-8073-e4f9427523dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47d891f8ea0c455b8879400164429ba8', 'request_id': 'req-a2841399-471e-46ed-8073-e4f9427523dd'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:48.069305 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-a2841399-471e-46ed-8073-e4f9427523dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47d891f8ea0c455b8879400164429ba8', 'request_id': 'req-a2841399-471e-46ed-8073-e4f9427523dd'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:48.158930 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-a2841399-471e-46ed-8073-e4f9427523dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47d891f8ea0c455b8879400164429ba8', 'request_id': 'req-a2841399-471e-46ed-8073-e4f9427523dd'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:48.170728 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-a2841399-471e-46ed-8073-e4f9427523dd admin admin] The token cache is being invalidate because project 47d891f8ea0c455b8879400164429ba8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 08:43:48.171013 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-a2841399-471e-46ed-8073-e4f9427523dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:48.171650 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-a2841399-471e-46ed-8073-e4f9427523dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:48.174012 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1063/2111] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 08:43:47 2022] DELETE /identity/v3/projects/47d891f8ea0c455b8879400164429ba8 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:43:48.177317 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b3733c-04be-445e-b017-34bd49aafb68 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:48.178707 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-33b3733c-04be-445e-b017-34bd49aafb68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:48.191700 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b3733c-04be-445e-b017-34bd49aafb68 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:48.248154 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc25d2c-f247-42f9-a1cb-df10f7f3a9a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:48.248769 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc25d2c-f247-42f9-a1cb-df10f7f3a9a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:48.249146 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc25d2c-f247-42f9-a1cb-df10f7f3a9a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:48.249416 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc25d2c-f247-42f9-a1cb-df10f7f3a9a8 admin admin] PATH_INFO: `/v3/users/acde448c012a4c6793379092795cb28f` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:48.250552 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc25d2c-f247-42f9-a1cb-df10f7f3a9a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=acde448c012a4c6793379092795cb28f)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:48.252457 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc25d2c-f247-42f9-a1cb-df10f7f3a9a8 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:48.505766 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-efc25d2c-f247-42f9-a1cb-df10f7f3a9a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'acde448c012a4c6793379092795cb28f'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:48.518432 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-efc25d2c-f247-42f9-a1cb-df10f7f3a9a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'acde448c012a4c6793379092795cb28f'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:48.525522 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-efc25d2c-f247-42f9-a1cb-df10f7f3a9a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'acde448c012a4c6793379092795cb28f'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:48.526695 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-efc25d2c-f247-42f9-a1cb-df10f7f3a9a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'acde448c012a4c6793379092795cb28f'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:48.527300 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-efc25d2c-f247-42f9-a1cb-df10f7f3a9a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'acde448c012a4c6793379092795cb28f'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:48.599225 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1049/2112] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 08:43:48 2022] DELETE /identity/v3/users/acde448c012a4c6793379092795cb28f => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 08:43:48.847850 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2841399-471e-46ed-8073-e4f9427523dd admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:48.851420 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-a2841399-471e-46ed-8073-e4f9427523dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:48.862764 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2841399-471e-46ed-8073-e4f9427523dd admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:48.884317 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:43:48.924466 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e6b0aa-2391-4731-b242-86d25dc1c6e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:48.925202 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e6b0aa-2391-4731-b242-86d25dc1c6e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:48.925483 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e6b0aa-2391-4731-b242-86d25dc1c6e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:48.925757 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e6b0aa-2391-4731-b242-86d25dc1c6e5 admin admin] PATH_INFO: `/v3/projects/dc73e10e90104bac9e3acfb07efcfe83` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:48.926881 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e6b0aa-2391-4731-b242-86d25dc1c6e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc73e10e90104bac9e3acfb07efcfe83)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:48.928830 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e6b0aa-2391-4731-b242-86d25dc1c6e5 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:49.046325 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-57e6b0aa-2391-4731-b242-86d25dc1c6e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc73e10e90104bac9e3acfb07efcfe83', 'request_id': 'req-57e6b0aa-2391-4731-b242-86d25dc1c6e5'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:49.051603 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-57e6b0aa-2391-4731-b242-86d25dc1c6e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc73e10e90104bac9e3acfb07efcfe83', 'request_id': 'req-57e6b0aa-2391-4731-b242-86d25dc1c6e5'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:49.134944 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-57e6b0aa-2391-4731-b242-86d25dc1c6e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc73e10e90104bac9e3acfb07efcfe83', 'request_id': 'req-57e6b0aa-2391-4731-b242-86d25dc1c6e5'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:49.147247 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-57e6b0aa-2391-4731-b242-86d25dc1c6e5 admin admin] The token cache is being invalidate because project dc73e10e90104bac9e3acfb07efcfe83 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 08:43:49.147535 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-57e6b0aa-2391-4731-b242-86d25dc1c6e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:49.148204 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-57e6b0aa-2391-4731-b242-86d25dc1c6e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:49.149781 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1064/2113] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 08:43:48 2022] DELETE /identity/v3/projects/dc73e10e90104bac9e3acfb07efcfe83 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:43:49.158432 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa001f3e-3d8e-4e36-abab-defc72e84a19 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:49.159457 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa001f3e-3d8e-4e36-abab-defc72e84a19 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:49.159737 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa001f3e-3d8e-4e36-abab-defc72e84a19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:49.172836 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24fe07c-85e7-486a-9896-81286087812b None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:49.173153 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24fe07c-85e7-486a-9896-81286087812b None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:49.173429 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24fe07c-85e7-486a-9896-81286087812b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:49.181480 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-aa001f3e-3d8e-4e36-abab-defc72e84a19 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:43:49.183390 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-aa001f3e-3d8e-4e36-abab-defc72e84a19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:49.192546 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-b24fe07c-85e7-486a-9896-81286087812b None None] MFA Rules not processed for user `c140d31bd2b84c48a170e3ca90a041ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:43:49.193747 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-b24fe07c-85e7-486a-9896-81286087812b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:49.227989 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:43:49.248549 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1050/2114] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:43:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:43:49.249404 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1065/2115] 10.222.0.57 () {58 vars in 1009 bytes} [Wed Aug 10 08:43:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 08:43:49.257818 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2fcfad-274c-488d-a602-e8c88238f5a5 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:49.257818 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2fcfad-274c-488d-a602-e8c88238f5a5 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:49.257818 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2fcfad-274c-488d-a602-e8c88238f5a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:49.279151 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-9b2fcfad-274c-488d-a602-e8c88238f5a5 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:43:49.280363 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-9b2fcfad-274c-488d-a602-e8c88238f5a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:49.288170 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1051/2116] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:43:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:43:49.292755 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24fe07c-85e7-486a-9896-81286087812b None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:49.293458 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2fcfad-274c-488d-a602-e8c88238f5a5 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:49.294832 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-9b2fcfad-274c-488d-a602-e8c88238f5a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:49.304334 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24fe07c-85e7-486a-9896-81286087812b None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:49.309639 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63345ee4-b8b5-4400-ac30-d57c3b7a92f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:49.310239 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63345ee4-b8b5-4400-ac30-d57c3b7a92f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:49.310513 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63345ee4-b8b5-4400-ac30-d57c3b7a92f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:49.310781 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63345ee4-b8b5-4400-ac30-d57c3b7a92f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:49.313770 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63345ee4-b8b5-4400-ac30-d57c3b7a92f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:43:49.313770 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63345ee4-b8b5-4400-ac30-d57c3b7a92f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:49.313770 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63345ee4-b8b5-4400-ac30-d57c3b7a92f8 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:49.326237 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1066/2117] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Aug 10 08:43:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:43:49.330410 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63345ee4-b8b5-4400-ac30-d57c3b7a92f8 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:49.340044 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63345ee4-b8b5-4400-ac30-d57c3b7a92f8 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:49.344714 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c3b367-40a2-40f1-af2e-0528a2f57964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:49.345330 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c3b367-40a2-40f1-af2e-0528a2f57964 admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:49.345605 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c3b367-40a2-40f1-af2e-0528a2f57964 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:49.345876 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c3b367-40a2-40f1-af2e-0528a2f57964 admin admin] PATH_INFO: `/v3/projects` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:49.346484 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c3b367-40a2-40f1-af2e-0528a2f57964 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:49.348377 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c3b367-40a2-40f1-af2e-0528a2f57964 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:49.364022 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2fcfad-274c-488d-a602-e8c88238f5a5 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:49.369049 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab2ee98-09e6-465d-a373-53d161a77ac1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4b26ba3a9e04c38b52db2a46198a53c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:49.370157 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab2ee98-09e6-465d-a373-53d161a77ac1 service swift] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:49.370442 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab2ee98-09e6-465d-a373-53d161a77ac1 service swift] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:49.370833 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab2ee98-09e6-465d-a373-53d161a77ac1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:49.379169 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab2ee98-09e6-465d-a373-53d161a77ac1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:49.380732 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab2ee98-09e6-465d-a373-53d161a77ac1 service swift] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:49.389591 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1052/2118] 10.222.0.57 () {60 vars in 1384 bytes} [Wed Aug 10 08:43:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 10 08:43:49.429594 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1067/2119] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 08:43:49 2022] POST /identity/v3/projects => generated 369 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:43:49.433944 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab2ee98-09e6-465d-a373-53d161a77ac1 service swift] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:49.444390 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab2ee98-09e6-465d-a373-53d161a77ac1 service swift] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:49.501689 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a312c8-db16-460b-a65e-2342c255f904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:49.502287 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a312c8-db16-460b-a65e-2342c255f904 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:49.502563 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a312c8-db16-460b-a65e-2342c255f904 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:49.502828 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a312c8-db16-460b-a65e-2342c255f904 admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:49.503435 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a312c8-db16-460b-a65e-2342c255f904 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:49.505362 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a312c8-db16-460b-a65e-2342c255f904 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:49.611329 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1053/2120] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 08:43:49 2022] POST /identity/v3/users => generated 285 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:43:49.615423 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c3b367-40a2-40f1-af2e-0528a2f57964 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:49.626269 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c3b367-40a2-40f1-af2e-0528a2f57964 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:49.630856 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f372b0-e8bf-4f92-bf34-412e966a1446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:49.631477 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f372b0-e8bf-4f92-bf34-412e966a1446 admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:49.631734 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f372b0-e8bf-4f92-bf34-412e966a1446 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:49.632006 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f372b0-e8bf-4f92-bf34-412e966a1446 admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:49.632641 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f372b0-e8bf-4f92-bf34-412e966a1446 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:49.634435 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f372b0-e8bf-4f92-bf34-412e966a1446 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:49.644742 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1068/2121] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:43:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:43:49.648765 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a312c8-db16-460b-a65e-2342c255f904 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:49.660580 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a312c8-db16-460b-a65e-2342c255f904 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:49.665719 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f8eecd-7647-4ee7-ac0a-87da15b09f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:49.666711 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f8eecd-7647-4ee7-ac0a-87da15b09f30 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:49.666920 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f8eecd-7647-4ee7-ac0a-87da15b09f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:49.667202 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f8eecd-7647-4ee7-ac0a-87da15b09f30 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:49.667885 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f8eecd-7647-4ee7-ac0a-87da15b09f30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:49.669489 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f8eecd-7647-4ee7-ac0a-87da15b09f30 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:49.680443 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1054/2122] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:43:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 08:43:49.684849 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f372b0-e8bf-4f92-bf34-412e966a1446 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:49.694962 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f372b0-e8bf-4f92-bf34-412e966a1446 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:49.700679 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00566a5-7caf-4940-a8ec-d9147de96559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:49.701160 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00566a5-7caf-4940-a8ec-d9147de96559 admin admin] REQUEST_METHOD: `PUT` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:49.701490 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00566a5-7caf-4940-a8ec-d9147de96559 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:49.701746 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00566a5-7caf-4940-a8ec-d9147de96559 admin admin] PATH_INFO: `/v3/projects/77c730a084314179a5a01b7ee2209a1f/users/1b8d1f020b444352b43fcd719f6ea797/roles/44b64db484f24a46bf0b7ced46b519bf` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:49.714461 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:43:49.718687 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00566a5-7caf-4940-a8ec-d9147de96559 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77c730a084314179a5a01b7ee2209a1f, user_id=1b8d1f020b444352b43fcd719f6ea797, role_id=44b64db484f24a46bf0b7ced46b519bf)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:49.721527 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00566a5-7caf-4940-a8ec-d9147de96559 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:49.786851 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1069/2123] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 08:43:49 2022] PUT /identity/v3/projects/77c730a084314179a5a01b7ee2209a1f/users/1b8d1f020b444352b43fcd719f6ea797/roles/44b64db484f24a46bf0b7ced46b519bf => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 08:43:49.797568 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ccfbba-ab68-432c-9c4d-76179fb3f3c1 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:49.797895 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ccfbba-ab68-432c-9c4d-76179fb3f3c1 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:49.798123 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ccfbba-ab68-432c-9c4d-76179fb3f3c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:49.828331 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-d4ccfbba-ab68-432c-9c4d-76179fb3f3c1 None None] MFA Rules not processed for user `1b8d1f020b444352b43fcd719f6ea797`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:43:49.829674 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-d4ccfbba-ab68-432c-9c4d-76179fb3f3c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:49.868616 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:43:49.903056 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1055/2124] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:43:49 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:43:49.911701 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00566a5-7caf-4940-a8ec-d9147de96559 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:49.983607 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00566a5-7caf-4940-a8ec-d9147de96559 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:49.988335 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da7a1cb-3d19-4010-a9f9-461fea5b16f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:49.988959 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da7a1cb-3d19-4010-a9f9-461fea5b16f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:49.989241 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da7a1cb-3d19-4010-a9f9-461fea5b16f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:49.989509 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da7a1cb-3d19-4010-a9f9-461fea5b16f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:49.990127 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da7a1cb-3d19-4010-a9f9-461fea5b16f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:49.992027 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da7a1cb-3d19-4010-a9f9-461fea5b16f5 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:50.111438 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1070/2125] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 08:43:49 2022] POST /identity/v3/projects => generated 369 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:43:50.117345 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ccfbba-ab68-432c-9c4d-76179fb3f3c1 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:50.128870 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ccfbba-ab68-432c-9c4d-76179fb3f3c1 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:50.184432 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8958c8-9a42-4a84-b2fc-8007fc19cf5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:50.185167 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8958c8-9a42-4a84-b2fc-8007fc19cf5b admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:50.185556 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8958c8-9a42-4a84-b2fc-8007fc19cf5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:50.185856 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8958c8-9a42-4a84-b2fc-8007fc19cf5b admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:50.186618 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8958c8-9a42-4a84-b2fc-8007fc19cf5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:50.189032 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8958c8-9a42-4a84-b2fc-8007fc19cf5b admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:50.215551 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21590708-4f49-4845-adf9-3cc07fa59392 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:50.218218 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21590708-4f49-4845-adf9-3cc07fa59392 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:50.218218 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21590708-4f49-4845-adf9-3cc07fa59392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:50.235354 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-21590708-4f49-4845-adf9-3cc07fa59392 None None] MFA Rules not processed for user `c140d31bd2b84c48a170e3ca90a041ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:43:50.236565 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-21590708-4f49-4845-adf9-3cc07fa59392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:50.271777 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1056/2126] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 08:43:50 2022] POST /identity/v3/users => generated 291 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:43:50.272840 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8958c8-9a42-4a84-b2fc-8007fc19cf5b admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:50.284148 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8958c8-9a42-4a84-b2fc-8007fc19cf5b admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:50.289019 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed468b28-0967-466b-b362-7b3eb4c6d205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:50.289618 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed468b28-0967-466b-b362-7b3eb4c6d205 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:50.289940 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed468b28-0967-466b-b362-7b3eb4c6d205 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:50.290246 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed468b28-0967-466b-b362-7b3eb4c6d205 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:50.290879 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed468b28-0967-466b-b362-7b3eb4c6d205 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:50.293162 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed468b28-0967-466b-b362-7b3eb4c6d205 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:50.299595 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1071/2127] 10.222.0.57 () {58 vars in 967 bytes} [Wed Aug 10 08:43:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 08:43:50.303503 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1057/2128] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:43:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:43:50.308848 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21590708-4f49-4845-adf9-3cc07fa59392 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:50.314157 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed468b28-0967-466b-b362-7b3eb4c6d205 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:50.320749 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21590708-4f49-4845-adf9-3cc07fa59392 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:50.326341 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92597c4b-85dc-426d-8688-12ce0a3aed27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:50.326995 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92597c4b-85dc-426d-8688-12ce0a3aed27 admin admin] REQUEST_METHOD: `PUT` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:50.327276 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92597c4b-85dc-426d-8688-12ce0a3aed27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:50.327552 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92597c4b-85dc-426d-8688-12ce0a3aed27 admin admin] PATH_INFO: `/v3/projects/89466ae6e3d645cd81b9a7e4f221e548/users/b86d416102854e799454a95d2eb14eb4/roles/ec3751692d9c4ecf81b4a71c936c46be` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:50.342506 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92597c4b-85dc-426d-8688-12ce0a3aed27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89466ae6e3d645cd81b9a7e4f221e548, user_id=b86d416102854e799454a95d2eb14eb4, role_id=ec3751692d9c4ecf81b4a71c936c46be)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:50.345224 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92597c4b-85dc-426d-8688-12ce0a3aed27 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:50.385333 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed468b28-0967-466b-b362-7b3eb4c6d205 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:50.390554 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1ec999-a631-479c-8d41-c5e8101f947b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4b26ba3a9e04c38b52db2a46198a53c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:50.391271 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1ec999-a631-479c-8d41-c5e8101f947b service swift] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:50.391548 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1ec999-a631-479c-8d41-c5e8101f947b service swift] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:50.391820 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1ec999-a631-479c-8d41-c5e8101f947b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:50.400218 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1ec999-a631-479c-8d41-c5e8101f947b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:50.402039 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1ec999-a631-479c-8d41-c5e8101f947b service swift] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:50.410372 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1058/2129] 10.222.0.57 () {60 vars in 1384 bytes} [Wed Aug 10 08:43:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 10 08:43:50.430531 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1072/2130] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 08:43:50 2022] PUT /identity/v3/projects/89466ae6e3d645cd81b9a7e4f221e548/users/b86d416102854e799454a95d2eb14eb4/roles/ec3751692d9c4ecf81b4a71c936c46be => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:43:50.434365 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1ec999-a631-479c-8d41-c5e8101f947b service swift] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:50.444877 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1ec999-a631-479c-8d41-c5e8101f947b service swift] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:50.486511 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:43:50.502197 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571852b0-8a6a-4062-b43b-86e4c914a7a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:50.502793 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571852b0-8a6a-4062-b43b-86e4c914a7a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:50.503073 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571852b0-8a6a-4062-b43b-86e4c914a7a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:50.503344 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571852b0-8a6a-4062-b43b-86e4c914a7a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:50.503999 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571852b0-8a6a-4062-b43b-86e4c914a7a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:50.505682 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571852b0-8a6a-4062-b43b-86e4c914a7a2 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:50.522153 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1059/2131] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:43:50 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:43:50.523432 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92597c4b-85dc-426d-8688-12ce0a3aed27 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:50.534050 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92597c4b-85dc-426d-8688-12ce0a3aed27 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:50.538754 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916186bc-d957-4278-b1b9-ce4149c1fcf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:50.539395 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916186bc-d957-4278-b1b9-ce4149c1fcf2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:50.539670 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916186bc-d957-4278-b1b9-ce4149c1fcf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:50.539938 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916186bc-d957-4278-b1b9-ce4149c1fcf2 admin admin] PATH_INFO: `/v3/projects/89466ae6e3d645cd81b9a7e4f221e548/users/b86d416102854e799454a95d2eb14eb4/roles/ec3751692d9c4ecf81b4a71c936c46be` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:50.542633 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916186bc-d957-4278-b1b9-ce4149c1fcf2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89466ae6e3d645cd81b9a7e4f221e548, user_id=b86d416102854e799454a95d2eb14eb4, role_id=ec3751692d9c4ecf81b4a71c936c46be)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:50.545305 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916186bc-d957-4278-b1b9-ce4149c1fcf2 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:50.555335 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1073/2132] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 08:43:50 2022] PUT /identity/v3/projects/89466ae6e3d645cd81b9a7e4f221e548/users/b86d416102854e799454a95d2eb14eb4/roles/ec3751692d9c4ecf81b4a71c936c46be => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:43:50.567293 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b832b1-b8cc-480e-a6f2-c1ab6c42730e None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:50.567584 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b832b1-b8cc-480e-a6f2-c1ab6c42730e None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:50.567858 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b832b1-b8cc-480e-a6f2-c1ab6c42730e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:50.596403 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-f9b832b1-b8cc-480e-a6f2-c1ab6c42730e None None] MFA Rules not processed for user `b86d416102854e799454a95d2eb14eb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:43:50.597675 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-f9b832b1-b8cc-480e-a6f2-c1ab6c42730e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:50.665201 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1060/2133] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:43:50 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:43:50.677484 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916186bc-d957-4278-b1b9-ce4149c1fcf2 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:50.679357 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-916186bc-d957-4278-b1b9-ce4149c1fcf2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:50.726525 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:43:50.737902 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916186bc-d957-4278-b1b9-ce4149c1fcf2 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:50.741651 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fea3ee8-761e-4b8a-8dea-73262e476cc5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ba74e28ced4cbba857f54222dfbe41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:50.742246 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fea3ee8-761e-4b8a-8dea-73262e476cc5 service cinder] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:50.742523 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fea3ee8-761e-4b8a-8dea-73262e476cc5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:50.742791 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fea3ee8-761e-4b8a-8dea-73262e476cc5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:50.752267 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fea3ee8-761e-4b8a-8dea-73262e476cc5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:50.753720 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fea3ee8-761e-4b8a-8dea-73262e476cc5 service cinder] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:50.764751 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1074/2134] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:43:50 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:43:50.891849 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b832b1-b8cc-480e-a6f2-c1ab6c42730e None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:50.893551 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-f9b832b1-b8cc-480e-a6f2-c1ab6c42730e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:50.906104 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b832b1-b8cc-480e-a6f2-c1ab6c42730e None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:50.911835 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9ad32c-f5c1-433b-b1e1-4ef706862e83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ba74e28ced4cbba857f54222dfbe41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:50.911835 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9ad32c-f5c1-433b-b1e1-4ef706862e83 service cinder] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:50.911835 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9ad32c-f5c1-433b-b1e1-4ef706862e83 service cinder] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:50.911835 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9ad32c-f5c1-433b-b1e1-4ef706862e83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:50.975666 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9ad32c-f5c1-433b-b1e1-4ef706862e83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:50.977239 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9ad32c-f5c1-433b-b1e1-4ef706862e83 service cinder] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:50.988049 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1061/2135] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:43:50 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:43:51.054994 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3714a7-88e7-4bdb-81ca-7187e13d730a None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:51.055438 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3714a7-88e7-4bdb-81ca-7187e13d730a None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:51.061257 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3714a7-88e7-4bdb-81ca-7187e13d730a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:51.075942 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-6c3714a7-88e7-4bdb-81ca-7187e13d730a None None] MFA Rules not processed for user `c140d31bd2b84c48a170e3ca90a041ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:43:51.077216 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-6c3714a7-88e7-4bdb-81ca-7187e13d730a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:51.133773 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1075/2136] 10.222.0.57 () {58 vars in 1009 bytes} [Wed Aug 10 08:43:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:43:51.147230 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9ad32c-f5c1-433b-b1e1-4ef706862e83 service cinder] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:51.217224 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9ad32c-f5c1-433b-b1e1-4ef706862e83 service cinder] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:51.222343 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455f2a82-f802-4f7e-83b7-a37efe7683d0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4b26ba3a9e04c38b52db2a46198a53c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:51.222944 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455f2a82-f802-4f7e-83b7-a37efe7683d0 service swift] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:51.223231 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455f2a82-f802-4f7e-83b7-a37efe7683d0 service swift] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:51.223495 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455f2a82-f802-4f7e-83b7-a37efe7683d0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:51.228586 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:43:51.233054 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455f2a82-f802-4f7e-83b7-a37efe7683d0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:51.234652 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455f2a82-f802-4f7e-83b7-a37efe7683d0 service swift] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:51.243075 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1062/2137] 10.222.0.57 () {60 vars in 1384 bytes} [Wed Aug 10 08:43:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 08:43:51.975750 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fbd195-3d8c-452f-b264-ce6a3a9c619d None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:51.976108 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fbd195-3d8c-452f-b264-ce6a3a9c619d None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:51.976454 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fbd195-3d8c-452f-b264-ce6a3a9c619d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:51.998726 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-f0fbd195-3d8c-452f-b264-ce6a3a9c619d None None] MFA Rules not processed for user `c140d31bd2b84c48a170e3ca90a041ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:43:52.000483 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-f0fbd195-3d8c-452f-b264-ce6a3a9c619d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:52.007938 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1076/2138] 10.222.0.57 () {58 vars in 967 bytes} [Wed Aug 10 08:43:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:43:52.022159 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455f2a82-f802-4f7e-83b7-a37efe7683d0 service swift] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:52.035276 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455f2a82-f802-4f7e-83b7-a37efe7683d0 service swift] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:52.043558 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25363871-9030-4cdb-890e-018ae23fcd43 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4b26ba3a9e04c38b52db2a46198a53c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:52.044830 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25363871-9030-4cdb-890e-018ae23fcd43 service swift] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:52.045523 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25363871-9030-4cdb-890e-018ae23fcd43 service swift] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:52.046477 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25363871-9030-4cdb-890e-018ae23fcd43 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:52.055811 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25363871-9030-4cdb-890e-018ae23fcd43 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:52.059664 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25363871-9030-4cdb-890e-018ae23fcd43 service swift] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:52.071201 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1063/2139] 10.222.0.57 () {60 vars in 1384 bytes} [Wed Aug 10 08:43:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 10 08:43:55.329865 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fbd195-3d8c-452f-b264-ce6a3a9c619d None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:55.331469 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-f0fbd195-3d8c-452f-b264-ce6a3a9c619d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:55.419707 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fbd195-3d8c-452f-b264-ce6a3a9c619d None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:55.424724 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a9030c-0244-4537-a4eb-4d6bd18587a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:55.425474 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a9030c-0244-4537-a4eb-4d6bd18587a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:55.425820 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a9030c-0244-4537-a4eb-4d6bd18587a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:55.426153 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a9030c-0244-4537-a4eb-4d6bd18587a8 admin admin] PATH_INFO: `/v3/users/28a3dda0c36a4b08aa2a314063005aa2` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:55.427460 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a9030c-0244-4537-a4eb-4d6bd18587a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=28a3dda0c36a4b08aa2a314063005aa2)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:55.429494 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a9030c-0244-4537-a4eb-4d6bd18587a8 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:55.624880 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-86a9030c-0244-4537-a4eb-4d6bd18587a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28a3dda0c36a4b08aa2a314063005aa2'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:55.637509 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-86a9030c-0244-4537-a4eb-4d6bd18587a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28a3dda0c36a4b08aa2a314063005aa2'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:55.644610 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-86a9030c-0244-4537-a4eb-4d6bd18587a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28a3dda0c36a4b08aa2a314063005aa2'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:55.707667 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-86a9030c-0244-4537-a4eb-4d6bd18587a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28a3dda0c36a4b08aa2a314063005aa2'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:55.708296 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-86a9030c-0244-4537-a4eb-4d6bd18587a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28a3dda0c36a4b08aa2a314063005aa2'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:55.714620 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1077/2140] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 08:43:55 2022] DELETE /identity/v3/users/28a3dda0c36a4b08aa2a314063005aa2 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:43:55.719602 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8210226a-8895-43fc-949a-8abc876de089 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:55.720104 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8210226a-8895-43fc-949a-8abc876de089 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:55.720386 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8210226a-8895-43fc-949a-8abc876de089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:55.740407 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-8210226a-8895-43fc-949a-8abc876de089 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:43:55.741692 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-8210226a-8895-43fc-949a-8abc876de089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:55.804592 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1064/2141] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:43:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 08:43:55.815320 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a9030c-0244-4537-a4eb-4d6bd18587a8 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:55.816736 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-86a9030c-0244-4537-a4eb-4d6bd18587a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:55.822164 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:43:55.874847 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a9030c-0244-4537-a4eb-4d6bd18587a8 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:55.879014 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e121698d-a334-45e9-af1c-e5e8b296d4c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:55.879667 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e121698d-a334-45e9-af1c-e5e8b296d4c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:55.879947 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e121698d-a334-45e9-af1c-e5e8b296d4c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:55.880219 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e121698d-a334-45e9-af1c-e5e8b296d4c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:55.889174 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e121698d-a334-45e9-af1c-e5e8b296d4c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:55.890607 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e121698d-a334-45e9-af1c-e5e8b296d4c2 service neutron] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:55.899842 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1078/2142] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:43:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:43:55.931425 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8210226a-8895-43fc-949a-8abc876de089 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:55.933010 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-8210226a-8895-43fc-949a-8abc876de089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:55.945386 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8210226a-8895-43fc-949a-8abc876de089 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:55.950696 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadfc6bb-3d5f-4398-9e75-53bfaed6829b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:55.951402 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadfc6bb-3d5f-4398-9e75-53bfaed6829b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:55.951684 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadfc6bb-3d5f-4398-9e75-53bfaed6829b admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:55.952048 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadfc6bb-3d5f-4398-9e75-53bfaed6829b admin admin] PATH_INFO: `/v3/users/1b8d1f020b444352b43fcd719f6ea797` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:55.953243 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadfc6bb-3d5f-4398-9e75-53bfaed6829b admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b8d1f020b444352b43fcd719f6ea797)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:55.955198 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadfc6bb-3d5f-4398-9e75-53bfaed6829b admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:56.144175 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-cadfc6bb-3d5f-4398-9e75-53bfaed6829b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b8d1f020b444352b43fcd719f6ea797'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:56.157588 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-cadfc6bb-3d5f-4398-9e75-53bfaed6829b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b8d1f020b444352b43fcd719f6ea797'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:56.165128 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-cadfc6bb-3d5f-4398-9e75-53bfaed6829b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b8d1f020b444352b43fcd719f6ea797'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:56.166093 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-cadfc6bb-3d5f-4398-9e75-53bfaed6829b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b8d1f020b444352b43fcd719f6ea797'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:56.166778 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-cadfc6bb-3d5f-4398-9e75-53bfaed6829b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b8d1f020b444352b43fcd719f6ea797'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:56.190525 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e121698d-a334-45e9-af1c-e5e8b296d4c2 service neutron] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:56.192847 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-e121698d-a334-45e9-af1c-e5e8b296d4c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:56.211178 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e121698d-a334-45e9-af1c-e5e8b296d4c2 service neutron] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:56.219033 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b0bd43-2e68-40c1-b26e-1499436d83fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:56.219620 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b0bd43-2e68-40c1-b26e-1499436d83fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:56.219874 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b0bd43-2e68-40c1-b26e-1499436d83fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:56.220146 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b0bd43-2e68-40c1-b26e-1499436d83fe admin admin] PATH_INFO: `/v3/projects/fa5c0f256e404409a5939c1d9e195b1c` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:56.221454 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b0bd43-2e68-40c1-b26e-1499436d83fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa5c0f256e404409a5939c1d9e195b1c)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:56.223308 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b0bd43-2e68-40c1-b26e-1499436d83fe admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:56.267463 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1065/2143] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 08:43:55 2022] DELETE /identity/v3/users/1b8d1f020b444352b43fcd719f6ea797 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:43:56.271781 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef49560-b369-4f7f-9f8f-cf00564d99c2 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:56.272068 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef49560-b369-4f7f-9f8f-cf00564d99c2 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:56.272340 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef49560-b369-4f7f-9f8f-cf00564d99c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:56.291974 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-1ef49560-b369-4f7f-9f8f-cf00564d99c2 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:43:56.293204 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-1ef49560-b369-4f7f-9f8f-cf00564d99c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:56.340466 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:43:56.349933 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-71b0bd43-2e68-40c1-b26e-1499436d83fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa5c0f256e404409a5939c1d9e195b1c', 'request_id': 'req-71b0bd43-2e68-40c1-b26e-1499436d83fe'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:56.353468 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1066/2144] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:43:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 08:43:56.355882 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-71b0bd43-2e68-40c1-b26e-1499436d83fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa5c0f256e404409a5939c1d9e195b1c', 'request_id': 'req-71b0bd43-2e68-40c1-b26e-1499436d83fe'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:56.364182 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef49560-b369-4f7f-9f8f-cf00564d99c2 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:56.365578 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-1ef49560-b369-4f7f-9f8f-cf00564d99c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:56.426605 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef49560-b369-4f7f-9f8f-cf00564d99c2 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:56.440048 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-71b0bd43-2e68-40c1-b26e-1499436d83fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa5c0f256e404409a5939c1d9e195b1c', 'request_id': 'req-71b0bd43-2e68-40c1-b26e-1499436d83fe'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:56.452356 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-71b0bd43-2e68-40c1-b26e-1499436d83fe admin admin] The token cache is being invalidate because project fa5c0f256e404409a5939c1d9e195b1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 08:43:56.452645 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-71b0bd43-2e68-40c1-b26e-1499436d83fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:56.453423 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-71b0bd43-2e68-40c1-b26e-1499436d83fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:56.454912 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1079/2145] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 08:43:56 2022] DELETE /identity/v3/projects/fa5c0f256e404409a5939c1d9e195b1c => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 08:43:56.460249 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df4a819-0639-4d19-bee8-874629077fc3 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:56.460596 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df4a819-0639-4d19-bee8-874629077fc3 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:56.460990 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df4a819-0639-4d19-bee8-874629077fc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:56.476416 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fffe1f9-df13-43cd-9876-ceabdb09ca01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:56.477096 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fffe1f9-df13-43cd-9876-ceabdb09ca01 service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:56.477384 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fffe1f9-df13-43cd-9876-ceabdb09ca01 service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:56.477660 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fffe1f9-df13-43cd-9876-ceabdb09ca01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:56.479568 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-8fffe1f9-df13-43cd-9876-ceabdb09ca01 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:56.479990 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-2df4a819-0639-4d19-bee8-874629077fc3 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:43:56.481199 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-2df4a819-0639-4d19-bee8-874629077fc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:56.543846 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1080/2146] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:43:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:43:56.547454 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fffe1f9-df13-43cd-9876-ceabdb09ca01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:56.548885 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fffe1f9-df13-43cd-9876-ceabdb09ca01 service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:56.552477 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87152bb8-cb69-4fd9-b27a-7da3e41db390 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:56.552987 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87152bb8-cb69-4fd9-b27a-7da3e41db390 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:56.553270 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87152bb8-cb69-4fd9-b27a-7da3e41db390 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:56.558786 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1067/2147] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:43:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 08:43:56.573887 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-87152bb8-cb69-4fd9-b27a-7da3e41db390 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:43:56.575169 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-87152bb8-cb69-4fd9-b27a-7da3e41db390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:56.584453 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1081/2148] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:43:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 08:43:56.588653 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fffe1f9-df13-43cd-9876-ceabdb09ca01 service neutron] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:56.598531 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fffe1f9-df13-43cd-9876-ceabdb09ca01 service neutron] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:56.603095 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bc6dca-bfd6-4c57-ab54-42d6311ea13c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:56.603752 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bc6dca-bfd6-4c57-ab54-42d6311ea13c admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:56.604028 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bc6dca-bfd6-4c57-ab54-42d6311ea13c admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:56.604355 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bc6dca-bfd6-4c57-ab54-42d6311ea13c admin admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:56.605031 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bc6dca-bfd6-4c57-ab54-42d6311ea13c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:43:56.605385 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bc6dca-bfd6-4c57-ab54-42d6311ea13c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:56.607537 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bc6dca-bfd6-4c57-ab54-42d6311ea13c admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:56.624406 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1068/2149] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Aug 10 08:43:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:43:56.625557 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87152bb8-cb69-4fd9-b27a-7da3e41db390 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:56.634697 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87152bb8-cb69-4fd9-b27a-7da3e41db390 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:56.640126 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3b46bf-224d-4f7b-bb1f-ca36236f7cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:56.640727 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3b46bf-224d-4f7b-bb1f-ca36236f7cd5 admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:56.641028 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3b46bf-224d-4f7b-bb1f-ca36236f7cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:56.641299 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3b46bf-224d-4f7b-bb1f-ca36236f7cd5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:56.641925 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3b46bf-224d-4f7b-bb1f-ca36236f7cd5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:56.643868 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3b46bf-224d-4f7b-bb1f-ca36236f7cd5 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:56.755336 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:43:56.766368 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1082/2150] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 08:43:56 2022] POST /identity/v3/projects => generated 373 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 08:43:56.769127 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bc6dca-bfd6-4c57-ab54-42d6311ea13c admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:56.780224 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bc6dca-bfd6-4c57-ab54-42d6311ea13c admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:56.826008 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:43:56.839956 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0929711d-c4d8-43a8-bc0a-ffe04a91eb7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:56.840575 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0929711d-c4d8-43a8-bc0a-ffe04a91eb7f admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:56.840860 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0929711d-c4d8-43a8-bc0a-ffe04a91eb7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:56.841168 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0929711d-c4d8-43a8-bc0a-ffe04a91eb7f admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:56.842332 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0929711d-c4d8-43a8-bc0a-ffe04a91eb7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:56.844307 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0929711d-c4d8-43a8-bc0a-ffe04a91eb7f admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:56.902513 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1069/2151] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 08:43:56 2022] POST /identity/v3/users => generated 287 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:43:56.907681 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3b46bf-224d-4f7b-bb1f-ca36236f7cd5 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:56.918493 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3b46bf-224d-4f7b-bb1f-ca36236f7cd5 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:56.924008 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844cfe31-6d48-495d-8683-6f70037a0336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:56.924604 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844cfe31-6d48-495d-8683-6f70037a0336 admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:56.924930 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844cfe31-6d48-495d-8683-6f70037a0336 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:56.925209 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844cfe31-6d48-495d-8683-6f70037a0336 admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:56.925843 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844cfe31-6d48-495d-8683-6f70037a0336 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:56.927432 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844cfe31-6d48-495d-8683-6f70037a0336 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:56.938029 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1083/2152] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:43:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:43:56.942328 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0929711d-c4d8-43a8-bc0a-ffe04a91eb7f admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:56.951475 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844cfe31-6d48-495d-8683-6f70037a0336 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:56.953537 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-844cfe31-6d48-495d-8683-6f70037a0336 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:56.954038 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0929711d-c4d8-43a8-bc0a-ffe04a91eb7f admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:56.959035 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7aedbf-0b08-42a2-b5db-30c7d4140943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:56.959662 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7aedbf-0b08-42a2-b5db-30c7d4140943 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:56.959938 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7aedbf-0b08-42a2-b5db-30c7d4140943 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:56.960204 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7aedbf-0b08-42a2-b5db-30c7d4140943 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:56.960849 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7aedbf-0b08-42a2-b5db-30c7d4140943 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:56.962459 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7aedbf-0b08-42a2-b5db-30c7d4140943 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:56.970537 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844cfe31-6d48-495d-8683-6f70037a0336 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:56.977854 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1070/2153] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:43:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:43:56.978286 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecee523-18e1-4d9b-b0c6-53a058d4e4c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:56.978286 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecee523-18e1-4d9b-b0c6-53a058d4e4c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:56.978286 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecee523-18e1-4d9b-b0c6-53a058d4e4c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:56.978736 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecee523-18e1-4d9b-b0c6-53a058d4e4c1 admin admin] PATH_INFO: `/v3/projects/77c730a084314179a5a01b7ee2209a1f` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:56.978929 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7aedbf-0b08-42a2-b5db-30c7d4140943 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:56.979996 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecee523-18e1-4d9b-b0c6-53a058d4e4c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77c730a084314179a5a01b7ee2209a1f)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:56.983137 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecee523-18e1-4d9b-b0c6-53a058d4e4c1 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:56.989964 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7aedbf-0b08-42a2-b5db-30c7d4140943 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:56.995055 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f9d990-fbcf-464a-8371-6545976ff8f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:56.995687 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f9d990-fbcf-464a-8371-6545976ff8f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:56.995961 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f9d990-fbcf-464a-8371-6545976ff8f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:56.996459 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f9d990-fbcf-464a-8371-6545976ff8f4 admin admin] PATH_INFO: `/v3/projects/370aff3310134a3fb2823908129ae45a/users/1d346511f25c4f059f0dedfb3467ed06/roles/44b64db484f24a46bf0b7ced46b519bf` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:57.014002 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f9d990-fbcf-464a-8371-6545976ff8f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=370aff3310134a3fb2823908129ae45a, user_id=1d346511f25c4f059f0dedfb3467ed06, role_id=44b64db484f24a46bf0b7ced46b519bf)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:57.017069 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f9d990-fbcf-464a-8371-6545976ff8f4 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:57.091151 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1071/2154] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 08:43:56 2022] PUT /identity/v3/projects/370aff3310134a3fb2823908129ae45a/users/1d346511f25c4f059f0dedfb3467ed06/roles/44b64db484f24a46bf0b7ced46b519bf => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:43:57.095606 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-4ecee523-18e1-4d9b-b0c6-53a058d4e4c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77c730a084314179a5a01b7ee2209a1f', 'request_id': 'req-4ecee523-18e1-4d9b-b0c6-53a058d4e4c1'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:57.096846 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19571f3-4a34-419a-8d89-d6c4d2139d38 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:57.098479 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19571f3-4a34-419a-8d89-d6c4d2139d38 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:57.098479 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19571f3-4a34-419a-8d89-d6c4d2139d38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:57.100971 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-4ecee523-18e1-4d9b-b0c6-53a058d4e4c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77c730a084314179a5a01b7ee2209a1f', 'request_id': 'req-4ecee523-18e1-4d9b-b0c6-53a058d4e4c1'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:57.116583 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-f19571f3-4a34-419a-8d89-d6c4d2139d38 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:43:57.117920 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-f19571f3-4a34-419a-8d89-d6c4d2139d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:57.164083 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-4ecee523-18e1-4d9b-b0c6-53a058d4e4c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77c730a084314179a5a01b7ee2209a1f', 'request_id': 'req-4ecee523-18e1-4d9b-b0c6-53a058d4e4c1'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:57.176657 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-4ecee523-18e1-4d9b-b0c6-53a058d4e4c1 admin admin] The token cache is being invalidate because project 77c730a084314179a5a01b7ee2209a1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 08:43:57.177019 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-4ecee523-18e1-4d9b-b0c6-53a058d4e4c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:57.177658 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-4ecee523-18e1-4d9b-b0c6-53a058d4e4c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:57.178910 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1072/2155] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:43:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 08:43:57.182158 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1084/2156] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 08:43:56 2022] DELETE /identity/v3/projects/77c730a084314179a5a01b7ee2209a1f => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 08:43:57.186122 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19571f3-4a34-419a-8d89-d6c4d2139d38 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:57.188477 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecee523-18e1-4d9b-b0c6-53a058d4e4c1 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:57.189937 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-4ecee523-18e1-4d9b-b0c6-53a058d4e4c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:57.195888 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19571f3-4a34-419a-8d89-d6c4d2139d38 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:57.201383 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87a5494-feda-4730-a29f-1494e069d0c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:57.202619 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87a5494-feda-4730-a29f-1494e069d0c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:57.202896 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87a5494-feda-4730-a29f-1494e069d0c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:57.203175 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87a5494-feda-4730-a29f-1494e069d0c2 admin admin] PATH_INFO: `/v3/users/b86d416102854e799454a95d2eb14eb4` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:57.204301 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87a5494-feda-4730-a29f-1494e069d0c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b86d416102854e799454a95d2eb14eb4)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:57.206228 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87a5494-feda-4730-a29f-1494e069d0c2 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:57.255992 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecee523-18e1-4d9b-b0c6-53a058d4e4c1 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:57.259758 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a192e169-334b-4ca5-a1c4-016aaf21e679 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:57.260363 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a192e169-334b-4ca5-a1c4-016aaf21e679 service neutron] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:57.260648 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a192e169-334b-4ca5-a1c4-016aaf21e679 service neutron] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:57.260935 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a192e169-334b-4ca5-a1c4-016aaf21e679 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:57.262969 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-a192e169-334b-4ca5-a1c4-016aaf21e679 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:57.275636 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a192e169-334b-4ca5-a1c4-016aaf21e679 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:57.277102 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a192e169-334b-4ca5-a1c4-016aaf21e679 service neutron] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:57.287585 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1085/2157] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:43:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 08:43:57.407360 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-c87a5494-feda-4730-a29f-1494e069d0c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b86d416102854e799454a95d2eb14eb4'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:57.419805 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-c87a5494-feda-4730-a29f-1494e069d0c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b86d416102854e799454a95d2eb14eb4'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:57.428888 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-c87a5494-feda-4730-a29f-1494e069d0c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b86d416102854e799454a95d2eb14eb4'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:57.429586 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-c87a5494-feda-4730-a29f-1494e069d0c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b86d416102854e799454a95d2eb14eb4'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:57.430222 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-c87a5494-feda-4730-a29f-1494e069d0c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b86d416102854e799454a95d2eb14eb4'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:57.509088 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1073/2158] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 08:43:57 2022] DELETE /identity/v3/users/b86d416102854e799454a95d2eb14eb4 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:43:58.001797 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a192e169-334b-4ca5-a1c4-016aaf21e679 service neutron] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:58.003270 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-a192e169-334b-4ca5-a1c4-016aaf21e679 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:58.016842 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a192e169-334b-4ca5-a1c4-016aaf21e679 service neutron] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:58.082533 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918e698a-8cfd-49fd-8d64-223aafb2837e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:58.083168 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918e698a-8cfd-49fd-8d64-223aafb2837e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:58.083447 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918e698a-8cfd-49fd-8d64-223aafb2837e admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:58.083721 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918e698a-8cfd-49fd-8d64-223aafb2837e admin admin] PATH_INFO: `/v3/projects/89466ae6e3d645cd81b9a7e4f221e548` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:58.084871 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918e698a-8cfd-49fd-8d64-223aafb2837e admin admin] RBAC: Authorizing `identity:delete_project(project_id=89466ae6e3d645cd81b9a7e4f221e548)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:58.086841 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918e698a-8cfd-49fd-8d64-223aafb2837e admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:58.103148 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:43:58.213793 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-918e698a-8cfd-49fd-8d64-223aafb2837e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89466ae6e3d645cd81b9a7e4f221e548', 'request_id': 'req-918e698a-8cfd-49fd-8d64-223aafb2837e'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:58.220074 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-918e698a-8cfd-49fd-8d64-223aafb2837e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89466ae6e3d645cd81b9a7e4f221e548', 'request_id': 'req-918e698a-8cfd-49fd-8d64-223aafb2837e'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:58.274156 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-918e698a-8cfd-49fd-8d64-223aafb2837e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89466ae6e3d645cd81b9a7e4f221e548', 'request_id': 'req-918e698a-8cfd-49fd-8d64-223aafb2837e'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:58.288069 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-918e698a-8cfd-49fd-8d64-223aafb2837e admin admin] The token cache is being invalidate because project 89466ae6e3d645cd81b9a7e4f221e548 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 08:43:58.288358 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-918e698a-8cfd-49fd-8d64-223aafb2837e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:58.289043 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-918e698a-8cfd-49fd-8d64-223aafb2837e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:43:58.290635 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1086/2159] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 08:43:58 2022] DELETE /identity/v3/projects/89466ae6e3d645cd81b9a7e4f221e548 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:43:58.296437 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2d109b-700d-4dc4-b3db-cf608a4d8d63 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:58.296966 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2d109b-700d-4dc4-b3db-cf608a4d8d63 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:58.297083 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2d109b-700d-4dc4-b3db-cf608a4d8d63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:58.317247 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-cb2d109b-700d-4dc4-b3db-cf608a4d8d63 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:43:58.318490 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-cb2d109b-700d-4dc4-b3db-cf608a4d8d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:58.327423 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:43:58.380144 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1074/2160] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:43:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:43:58.388204 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a64a12d-75e8-4706-8c5f-8fec50726fac None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:58.389197 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a64a12d-75e8-4706-8c5f-8fec50726fac None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:58.389360 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a64a12d-75e8-4706-8c5f-8fec50726fac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:58.413103 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-0a64a12d-75e8-4706-8c5f-8fec50726fac None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:43:58.413671 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-0a64a12d-75e8-4706-8c5f-8fec50726fac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:58.425574 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1087/2161] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:43:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:43:58.429227 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2d109b-700d-4dc4-b3db-cf608a4d8d63 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:58.439322 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2d109b-700d-4dc4-b3db-cf608a4d8d63 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:58.444418 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d64569-a363-4008-a033-333735a58120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:58.445076 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d64569-a363-4008-a033-333735a58120 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:58.445359 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d64569-a363-4008-a033-333735a58120 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:58.445624 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d64569-a363-4008-a033-333735a58120 admin admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:58.446268 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d64569-a363-4008-a033-333735a58120 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:43:58.446569 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d64569-a363-4008-a033-333735a58120 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:58.448178 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d64569-a363-4008-a033-333735a58120 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:58.460949 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1075/2162] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Aug 10 08:43:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:43:58.464850 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a64a12d-75e8-4706-8c5f-8fec50726fac None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:58.475053 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a64a12d-75e8-4706-8c5f-8fec50726fac None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:58.480133 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bdf346-abbe-4753-af40-dcbdee68532c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:58.481057 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bdf346-abbe-4753-af40-dcbdee68532c admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:58.481421 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bdf346-abbe-4753-af40-dcbdee68532c admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:58.481758 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bdf346-abbe-4753-af40-dcbdee68532c admin admin] PATH_INFO: `/v3/projects` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:58.483038 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bdf346-abbe-4753-af40-dcbdee68532c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:58.485228 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bdf346-abbe-4753-af40-dcbdee68532c admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:58.616721 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1088/2163] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 08:43:58 2022] POST /identity/v3/projects => generated 383 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:43:58.621004 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d64569-a363-4008-a033-333735a58120 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:58.631634 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d64569-a363-4008-a033-333735a58120 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:58.699506 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f219fb-5c91-45c2-9a01-db1f4b04df0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:58.700192 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f219fb-5c91-45c2-9a01-db1f4b04df0f admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:58.700466 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f219fb-5c91-45c2-9a01-db1f4b04df0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:58.700734 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f219fb-5c91-45c2-9a01-db1f4b04df0f admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:58.702581 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f219fb-5c91-45c2-9a01-db1f4b04df0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:58.704523 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f219fb-5c91-45c2-9a01-db1f4b04df0f admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:58.820535 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1076/2164] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 08:43:58 2022] POST /identity/v3/users => generated 292 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:43:58.821744 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bdf346-abbe-4753-af40-dcbdee68532c admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:58.833500 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bdf346-abbe-4753-af40-dcbdee68532c admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:58.838994 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d7ffa5-2e3d-4fbe-8a3b-066a4ecb4301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:58.839600 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d7ffa5-2e3d-4fbe-8a3b-066a4ecb4301 admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:58.839882 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d7ffa5-2e3d-4fbe-8a3b-066a4ecb4301 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:58.840152 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d7ffa5-2e3d-4fbe-8a3b-066a4ecb4301 admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:58.840800 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d7ffa5-2e3d-4fbe-8a3b-066a4ecb4301 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:58.843067 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d7ffa5-2e3d-4fbe-8a3b-066a4ecb4301 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:58.853287 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1089/2165] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:43:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:43:58.858127 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f219fb-5c91-45c2-9a01-db1f4b04df0f admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:58.868351 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f219fb-5c91-45c2-9a01-db1f4b04df0f admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:58.874395 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09035ab5-a231-4d8e-9685-b82cb3afdab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:58.874992 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09035ab5-a231-4d8e-9685-b82cb3afdab6 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:58.875382 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09035ab5-a231-4d8e-9685-b82cb3afdab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:58.875653 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09035ab5-a231-4d8e-9685-b82cb3afdab6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:58.876290 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09035ab5-a231-4d8e-9685-b82cb3afdab6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:58.877899 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09035ab5-a231-4d8e-9685-b82cb3afdab6 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:58.889923 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1077/2166] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:43:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:43:58.893927 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d7ffa5-2e3d-4fbe-8a3b-066a4ecb4301 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:58.906210 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d7ffa5-2e3d-4fbe-8a3b-066a4ecb4301 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:58.911526 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a76e638-0352-42fe-b334-172508ec770e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:58.912156 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a76e638-0352-42fe-b334-172508ec770e admin admin] REQUEST_METHOD: `PUT` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:58.912450 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a76e638-0352-42fe-b334-172508ec770e admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:58.912700 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a76e638-0352-42fe-b334-172508ec770e admin admin] PATH_INFO: `/v3/projects/80b6d43aad594e56ad099718bbba8e79/users/fbbf94b25df444c4a767cf1a5e4b1fb1/roles/44b64db484f24a46bf0b7ced46b519bf` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:58.931880 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a76e638-0352-42fe-b334-172508ec770e admin admin] RBAC: Authorizing `identity:create_grant(project_id=80b6d43aad594e56ad099718bbba8e79, user_id=fbbf94b25df444c4a767cf1a5e4b1fb1, role_id=44b64db484f24a46bf0b7ced46b519bf)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:58.934819 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a76e638-0352-42fe-b334-172508ec770e admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:59.077571 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1090/2167] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 08:43:58 2022] PUT /identity/v3/projects/80b6d43aad594e56ad099718bbba8e79/users/fbbf94b25df444c4a767cf1a5e4b1fb1/roles/44b64db484f24a46bf0b7ced46b519bf => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 08:43:59.088198 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321958fc-48d1-4a50-a141-f0a4c7c662e1 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:59.088590 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321958fc-48d1-4a50-a141-f0a4c7c662e1 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:59.088956 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321958fc-48d1-4a50-a141-f0a4c7c662e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:59.118575 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-321958fc-48d1-4a50-a141-f0a4c7c662e1 None None] MFA Rules not processed for user `fbbf94b25df444c4a767cf1a5e4b1fb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:43:59.119898 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-321958fc-48d1-4a50-a141-f0a4c7c662e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:43:59.192716 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1078/2168] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:43:59 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:43:59.197515 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a76e638-0352-42fe-b334-172508ec770e admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:59.271740 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a76e638-0352-42fe-b334-172508ec770e admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:59.276691 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240b04e7-f18b-41a5-87cd-dc4e7f5212b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:59.277329 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240b04e7-f18b-41a5-87cd-dc4e7f5212b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:59.277612 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240b04e7-f18b-41a5-87cd-dc4e7f5212b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:59.277991 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240b04e7-f18b-41a5-87cd-dc4e7f5212b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:59.278515 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240b04e7-f18b-41a5-87cd-dc4e7f5212b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:59.280614 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240b04e7-f18b-41a5-87cd-dc4e7f5212b5 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:59.384951 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1091/2169] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 08:43:59 2022] POST /identity/v3/projects => generated 381 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:43:59.389356 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321958fc-48d1-4a50-a141-f0a4c7c662e1 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:59.400736 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321958fc-48d1-4a50-a141-f0a4c7c662e1 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:59.420784 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:43:59.498609 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665e08be-b779-457d-85a7-70123715b253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:59.499557 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665e08be-b779-457d-85a7-70123715b253 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:59.500035 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665e08be-b779-457d-85a7-70123715b253 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:59.500309 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665e08be-b779-457d-85a7-70123715b253 admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:59.501128 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665e08be-b779-457d-85a7-70123715b253 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:59.503126 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665e08be-b779-457d-85a7-70123715b253 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:59.613778 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1079/2170] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 08:43:59 2022] POST /identity/v3/users => generated 297 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:43:59.614829 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240b04e7-f18b-41a5-87cd-dc4e7f5212b5 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:59.651007 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240b04e7-f18b-41a5-87cd-dc4e7f5212b5 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:59.670778 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eec454-09a9-4b80-ac79-d636db44cc87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:59.685952 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eec454-09a9-4b80-ac79-d636db44cc87 admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:59.685952 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eec454-09a9-4b80-ac79-d636db44cc87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:59.685952 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eec454-09a9-4b80-ac79-d636db44cc87 admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:59.685952 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88eec454-09a9-4b80-ac79-d636db44cc87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:59.685952 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88eec454-09a9-4b80-ac79-d636db44cc87 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:59.707202 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1092/2171] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:43:59 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:43:59.714245 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665e08be-b779-457d-85a7-70123715b253 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:59.721328 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665e08be-b779-457d-85a7-70123715b253 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:59.730071 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e78a74-1137-4c40-9bc6-cd4189404076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:59.730971 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e78a74-1137-4c40-9bc6-cd4189404076 admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:59.731249 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e78a74-1137-4c40-9bc6-cd4189404076 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:59.731519 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e78a74-1137-4c40-9bc6-cd4189404076 admin admin] PATH_INFO: `/v3/projects/4477f89809db4de8b62f16f570e96f4a/users/054a6e04c16c4e5690c083a18c4b80e8/roles/ec3751692d9c4ecf81b4a71c936c46be` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:59.750611 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e78a74-1137-4c40-9bc6-cd4189404076 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4477f89809db4de8b62f16f570e96f4a, user_id=054a6e04c16c4e5690c083a18c4b80e8, role_id=ec3751692d9c4ecf81b4a71c936c46be)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:59.758368 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e78a74-1137-4c40-9bc6-cd4189404076 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:59.848717 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1080/2172] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 08:43:59 2022] PUT /identity/v3/projects/4477f89809db4de8b62f16f570e96f4a/users/054a6e04c16c4e5690c083a18c4b80e8/roles/ec3751692d9c4ecf81b4a71c936c46be => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:43:59.852665 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eec454-09a9-4b80-ac79-d636db44cc87 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:59.864597 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eec454-09a9-4b80-ac79-d636db44cc87 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:59.878927 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:43:59.927904 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e5c751-8afb-4e43-95b2-7966134ead64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:59.928571 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e5c751-8afb-4e43-95b2-7966134ead64 admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:59.928849 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e5c751-8afb-4e43-95b2-7966134ead64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:59.929167 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e5c751-8afb-4e43-95b2-7966134ead64 admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:59.929851 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e5c751-8afb-4e43-95b2-7966134ead64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:59.931568 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e5c751-8afb-4e43-95b2-7966134ead64 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:59.945842 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1093/2173] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:43:59 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:43:59.951383 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e78a74-1137-4c40-9bc6-cd4189404076 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:43:59.957781 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e78a74-1137-4c40-9bc6-cd4189404076 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:43:59.962355 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf4da75-e0aa-41eb-bdff-0be2a1a4829c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:43:59.962999 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf4da75-e0aa-41eb-bdff-0be2a1a4829c admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:59.963274 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf4da75-e0aa-41eb-bdff-0be2a1a4829c admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:59.963548 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf4da75-e0aa-41eb-bdff-0be2a1a4829c admin admin] PATH_INFO: `/v3/projects/4477f89809db4de8b62f16f570e96f4a/users/054a6e04c16c4e5690c083a18c4b80e8/roles/ec3751692d9c4ecf81b4a71c936c46be` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:43:59.965575 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf4da75-e0aa-41eb-bdff-0be2a1a4829c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4477f89809db4de8b62f16f570e96f4a, user_id=054a6e04c16c4e5690c083a18c4b80e8, role_id=ec3751692d9c4ecf81b4a71c936c46be)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:43:59.969380 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf4da75-e0aa-41eb-bdff-0be2a1a4829c admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:43:59.993542 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1081/2174] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 08:43:59 2022] PUT /identity/v3/projects/4477f89809db4de8b62f16f570e96f4a/users/054a6e04c16c4e5690c083a18c4b80e8/roles/ec3751692d9c4ecf81b4a71c936c46be => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:43:59.997495 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbba690a-9425-432f-a414-03e7137835fe None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:43:59.998079 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbba690a-9425-432f-a414-03e7137835fe None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:43:59.998357 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbba690a-9425-432f-a414-03e7137835fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:00.027173 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-fbba690a-9425-432f-a414-03e7137835fe None None] MFA Rules not processed for user `054a6e04c16c4e5690c083a18c4b80e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:44:00.028385 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-fbba690a-9425-432f-a414-03e7137835fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:44:00.107900 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1094/2175] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:43:59 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:44:00.120616 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf4da75-e0aa-41eb-bdff-0be2a1a4829c admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:44:00.123125 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-fbf4da75-e0aa-41eb-bdff-0be2a1a4829c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:44:00.194820 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf4da75-e0aa-41eb-bdff-0be2a1a4829c admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:44:00.200359 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db814a04-d2bd-401d-8174-d1f5c8f4dd70 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ba74e28ced4cbba857f54222dfbe41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:44:00.201702 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db814a04-d2bd-401d-8174-d1f5c8f4dd70 service cinder] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:00.202056 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db814a04-d2bd-401d-8174-d1f5c8f4dd70 service cinder] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:00.202549 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db814a04-d2bd-401d-8174-d1f5c8f4dd70 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:00.212540 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db814a04-d2bd-401d-8174-d1f5c8f4dd70 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:44:00.214114 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db814a04-d2bd-401d-8174-d1f5c8f4dd70 service cinder] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:44:00.225986 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1082/2176] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:44:00 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:44:00.336443 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbba690a-9425-432f-a414-03e7137835fe None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:44:00.347424 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbba690a-9425-432f-a414-03e7137835fe None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:44:00.354919 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e91bbf4-04e2-4568-a750-9e498dcc7109 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ba74e28ced4cbba857f54222dfbe41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:44:00.356708 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e91bbf4-04e2-4568-a750-9e498dcc7109 service cinder] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:00.356708 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e91bbf4-04e2-4568-a750-9e498dcc7109 service cinder] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:00.356708 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e91bbf4-04e2-4568-a750-9e498dcc7109 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:00.455162 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e91bbf4-04e2-4568-a750-9e498dcc7109 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:44:00.457094 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e91bbf4-04e2-4568-a750-9e498dcc7109 service cinder] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:44:00.468975 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1095/2177] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:44:00 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 08:44:03.302849 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed395b5-555c-40ad-a51d-f91069134571 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:03.303480 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed395b5-555c-40ad-a51d-f91069134571 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:03.303480 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed395b5-555c-40ad-a51d-f91069134571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:03.338291 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-9ed395b5-555c-40ad-a51d-f91069134571 None None] MFA Rules not processed for user `1d346511f25c4f059f0dedfb3467ed06`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:44:03.340352 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-9ed395b5-555c-40ad-a51d-f91069134571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:44:03.454012 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:44:03.467785 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1083/2178] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:44:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:44:03.471663 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e91bbf4-04e2-4568-a750-9e498dcc7109 service cinder] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:44:03.473645 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-2e91bbf4-04e2-4568-a750-9e498dcc7109 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:44:03.534209 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:44:03.572499 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e91bbf4-04e2-4568-a750-9e498dcc7109 service cinder] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:44:03.579851 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b8a3c7-4896-418f-b334-84320ef990a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:44:03.580635 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b8a3c7-4896-418f-b334-84320ef990a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:03.581372 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b8a3c7-4896-418f-b334-84320ef990a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:03.581949 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b8a3c7-4896-418f-b334-84320ef990a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:03.582729 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b8a3c7-4896-418f-b334-84320ef990a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:44:03.585068 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b8a3c7-4896-418f-b334-84320ef990a9 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:44:03.725001 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1096/2179] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 08:44:03 2022] POST /identity/v3/projects => generated 371 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:44:03.730224 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed395b5-555c-40ad-a51d-f91069134571 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:44:03.744615 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed395b5-555c-40ad-a51d-f91069134571 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:44:03.828478 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262e4688-fc45-4902-973b-2954ab660ca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:44:03.829456 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262e4688-fc45-4902-973b-2954ab660ca9 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:03.829799 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262e4688-fc45-4902-973b-2954ab660ca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:03.835882 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262e4688-fc45-4902-973b-2954ab660ca9 admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:03.835882 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262e4688-fc45-4902-973b-2954ab660ca9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:44:03.835882 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262e4688-fc45-4902-973b-2954ab660ca9 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:44:03.976660 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1084/2180] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 08:44:03 2022] POST /identity/v3/users => generated 292 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:44:03.977908 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b8a3c7-4896-418f-b334-84320ef990a9 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:44:03.989168 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b8a3c7-4896-418f-b334-84320ef990a9 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:44:03.994024 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaeaa76-7629-4807-931e-acabc31f42de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:44:03.994633 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaeaa76-7629-4807-931e-acabc31f42de admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:03.994914 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaeaa76-7629-4807-931e-acabc31f42de admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:03.995182 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaeaa76-7629-4807-931e-acabc31f42de admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:03.995815 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffaeaa76-7629-4807-931e-acabc31f42de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:44:03.997432 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffaeaa76-7629-4807-931e-acabc31f42de admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:44:04.008652 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1097/2181] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:44:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:44:04.013245 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262e4688-fc45-4902-973b-2954ab660ca9 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:44:04.023981 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262e4688-fc45-4902-973b-2954ab660ca9 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:44:04.028835 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b0ca6c-cc2b-48fb-b8da-be078e1022c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:44:04.030323 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b0ca6c-cc2b-48fb-b8da-be078e1022c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:04.030744 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b0ca6c-cc2b-48fb-b8da-be078e1022c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:04.031079 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b0ca6c-cc2b-48fb-b8da-be078e1022c8 admin admin] PATH_INFO: `/v3/projects/b2ca359757dd4369911e58622104c26d/users/f553fa4ae3ca41d68d0872e13d8161cd/roles/ec3751692d9c4ecf81b4a71c936c46be` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:04.048634 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b0ca6c-cc2b-48fb-b8da-be078e1022c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2ca359757dd4369911e58622104c26d, user_id=f553fa4ae3ca41d68d0872e13d8161cd, role_id=ec3751692d9c4ecf81b4a71c936c46be)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:44:04.051379 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b0ca6c-cc2b-48fb-b8da-be078e1022c8 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:44:04.109716 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1085/2182] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 08:44:04 2022] PUT /identity/v3/projects/b2ca359757dd4369911e58622104c26d/users/f553fa4ae3ca41d68d0872e13d8161cd/roles/ec3751692d9c4ecf81b4a71c936c46be => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:44:04.113772 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaeaa76-7629-4807-931e-acabc31f42de admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:44:04.124743 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaeaa76-7629-4807-931e-acabc31f42de admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:44:04.182990 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af767f45-a8cd-4037-9b96-5f3b9fbbfc9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:44:04.183606 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af767f45-a8cd-4037-9b96-5f3b9fbbfc9c admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:04.183878 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af767f45-a8cd-4037-9b96-5f3b9fbbfc9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:04.184153 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af767f45-a8cd-4037-9b96-5f3b9fbbfc9c admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:04.184820 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af767f45-a8cd-4037-9b96-5f3b9fbbfc9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:44:04.187069 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af767f45-a8cd-4037-9b96-5f3b9fbbfc9c admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:44:04.196762 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1098/2183] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:44:04 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 08:44:04.202428 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b0ca6c-cc2b-48fb-b8da-be078e1022c8 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:44:04.213434 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b0ca6c-cc2b-48fb-b8da-be078e1022c8 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:44:04.218396 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d4d906-35fb-4821-b9fa-8b2039f6d8f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:44:04.219000 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d4d906-35fb-4821-b9fa-8b2039f6d8f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:04.219278 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d4d906-35fb-4821-b9fa-8b2039f6d8f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:04.219552 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d4d906-35fb-4821-b9fa-8b2039f6d8f4 admin admin] PATH_INFO: `/v3/projects/b2ca359757dd4369911e58622104c26d/users/f553fa4ae3ca41d68d0872e13d8161cd/roles/ec3751692d9c4ecf81b4a71c936c46be` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:04.231583 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d4d906-35fb-4821-b9fa-8b2039f6d8f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2ca359757dd4369911e58622104c26d, user_id=f553fa4ae3ca41d68d0872e13d8161cd, role_id=ec3751692d9c4ecf81b4a71c936c46be)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:44:04.231583 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d4d906-35fb-4821-b9fa-8b2039f6d8f4 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:44:04.236452 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1086/2184] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 08:44:04 2022] PUT /identity/v3/projects/b2ca359757dd4369911e58622104c26d/users/f553fa4ae3ca41d68d0872e13d8161cd/roles/ec3751692d9c4ecf81b4a71c936c46be => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:44:04.804672 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af767f45-a8cd-4037-9b96-5f3b9fbbfc9c admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:44:04.817568 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af767f45-a8cd-4037-9b96-5f3b9fbbfc9c admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:44:04.955670 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf424ef4-6860-4739-bd2b-fd8233b1823b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:44:04.960009 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf424ef4-6860-4739-bd2b-fd8233b1823b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:04.960908 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf424ef4-6860-4739-bd2b-fd8233b1823b admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:04.960908 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf424ef4-6860-4739-bd2b-fd8233b1823b admin admin] PATH_INFO: `/v3/users/fbbf94b25df444c4a767cf1a5e4b1fb1` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:04.961875 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf424ef4-6860-4739-bd2b-fd8233b1823b admin admin] RBAC: Authorizing `identity:delete_user(user_id=fbbf94b25df444c4a767cf1a5e4b1fb1)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:44:04.963895 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf424ef4-6860-4739-bd2b-fd8233b1823b admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:44:05.140553 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-bf424ef4-6860-4739-bd2b-fd8233b1823b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fbbf94b25df444c4a767cf1a5e4b1fb1'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:44:05.142010 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:44:05.154004 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-bf424ef4-6860-4739-bd2b-fd8233b1823b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fbbf94b25df444c4a767cf1a5e4b1fb1'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:44:05.161483 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-bf424ef4-6860-4739-bd2b-fd8233b1823b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fbbf94b25df444c4a767cf1a5e4b1fb1'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:44:05.264354 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-bf424ef4-6860-4739-bd2b-fd8233b1823b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fbbf94b25df444c4a767cf1a5e4b1fb1'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:44:05.265620 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-bf424ef4-6860-4739-bd2b-fd8233b1823b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fbbf94b25df444c4a767cf1a5e4b1fb1'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:44:05.272033 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1099/2185] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 08:44:04 2022] DELETE /identity/v3/users/fbbf94b25df444c4a767cf1a5e4b1fb1 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:44:05.276528 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d8abb2-8010-4736-b02a-2b5ead42b827 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:05.277092 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d8abb2-8010-4736-b02a-2b5ead42b827 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:05.277192 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d8abb2-8010-4736-b02a-2b5ead42b827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:05.297649 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-b7d8abb2-8010-4736-b02a-2b5ead42b827 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:44:05.298908 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-b7d8abb2-8010-4736-b02a-2b5ead42b827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:44:05.359749 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1087/2186] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:44:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:44:05.369331 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf424ef4-6860-4739-bd2b-fd8233b1823b admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:44:05.370731 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-bf424ef4-6860-4739-bd2b-fd8233b1823b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:44:05.431997 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf424ef4-6860-4739-bd2b-fd8233b1823b admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:44:05.436143 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf96a59-75d0-4d56-850c-ce5669cd98ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:44:05.437404 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf96a59-75d0-4d56-850c-ce5669cd98ae service neutron] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:05.437684 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf96a59-75d0-4d56-850c-ce5669cd98ae service neutron] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:05.437952 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf96a59-75d0-4d56-850c-ce5669cd98ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:05.447005 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf96a59-75d0-4d56-850c-ce5669cd98ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:44:05.448422 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf96a59-75d0-4d56-850c-ce5669cd98ae service neutron] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:44:05.459010 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1100/2187] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:44:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:44:06.005363 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d8abb2-8010-4736-b02a-2b5ead42b827 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:44:06.006826 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-b7d8abb2-8010-4736-b02a-2b5ead42b827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:44:06.027705 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d8abb2-8010-4736-b02a-2b5ead42b827 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:44:06.041291 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1834f6-b19e-4824-bf3e-6a4eda7212a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:44:06.041954 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1834f6-b19e-4824-bf3e-6a4eda7212a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:06.042236 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1834f6-b19e-4824-bf3e-6a4eda7212a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:06.042509 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1834f6-b19e-4824-bf3e-6a4eda7212a3 admin admin] PATH_INFO: `/v3/projects/80b6d43aad594e56ad099718bbba8e79` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:06.043775 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1834f6-b19e-4824-bf3e-6a4eda7212a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80b6d43aad594e56ad099718bbba8e79)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:44:06.046839 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1834f6-b19e-4824-bf3e-6a4eda7212a3 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:44:06.223531 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:44:06.235294 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-dd1834f6-b19e-4824-bf3e-6a4eda7212a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80b6d43aad594e56ad099718bbba8e79', 'request_id': 'req-dd1834f6-b19e-4824-bf3e-6a4eda7212a3'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:44:06.318899 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-dd1834f6-b19e-4824-bf3e-6a4eda7212a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80b6d43aad594e56ad099718bbba8e79', 'request_id': 'req-dd1834f6-b19e-4824-bf3e-6a4eda7212a3'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:44:06.329885 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-dd1834f6-b19e-4824-bf3e-6a4eda7212a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80b6d43aad594e56ad099718bbba8e79', 'request_id': 'req-dd1834f6-b19e-4824-bf3e-6a4eda7212a3'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:44:06.381745 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-dd1834f6-b19e-4824-bf3e-6a4eda7212a3 admin admin] The token cache is being invalidate because project 80b6d43aad594e56ad099718bbba8e79 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 08:44:06.382232 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-dd1834f6-b19e-4824-bf3e-6a4eda7212a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:44:06.383463 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-dd1834f6-b19e-4824-bf3e-6a4eda7212a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:44:06.385313 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1088/2188] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 08:44:06 2022] DELETE /identity/v3/projects/80b6d43aad594e56ad099718bbba8e79 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:44:06.391040 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf96a59-75d0-4d56-850c-ce5669cd98ae service neutron] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:44:06.432629 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf96a59-75d0-4d56-850c-ce5669cd98ae service neutron] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:44:06.528207 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7c15dc-e647-4452-a049-72c4c6293493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:44:06.540012 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7c15dc-e647-4452-a049-72c4c6293493 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:06.540226 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7c15dc-e647-4452-a049-72c4c6293493 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:06.540460 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7c15dc-e647-4452-a049-72c4c6293493 admin admin] PATH_INFO: `/v3/users/054a6e04c16c4e5690c083a18c4b80e8` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:06.549944 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7c15dc-e647-4452-a049-72c4c6293493 admin admin] RBAC: Authorizing `identity:delete_user(user_id=054a6e04c16c4e5690c083a18c4b80e8)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:44:06.552397 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7c15dc-e647-4452-a049-72c4c6293493 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:44:06.851340 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-8f7c15dc-e647-4452-a049-72c4c6293493 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '054a6e04c16c4e5690c083a18c4b80e8'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:44:06.870554 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-8f7c15dc-e647-4452-a049-72c4c6293493 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '054a6e04c16c4e5690c083a18c4b80e8'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:44:06.883660 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:44:06.887875 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-8f7c15dc-e647-4452-a049-72c4c6293493 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '054a6e04c16c4e5690c083a18c4b80e8'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:44:06.971847 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-8f7c15dc-e647-4452-a049-72c4c6293493 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '054a6e04c16c4e5690c083a18c4b80e8'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:44:06.972742 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-8f7c15dc-e647-4452-a049-72c4c6293493 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '054a6e04c16c4e5690c083a18c4b80e8'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:44:06.983386 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1101/2189] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 08:44:06 2022] DELETE /identity/v3/users/054a6e04c16c4e5690c083a18c4b80e8 => generated 0 bytes in 593 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:44:07.528674 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1834f6-b19e-4824-bf3e-6a4eda7212a3 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:44:07.534269 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-dd1834f6-b19e-4824-bf3e-6a4eda7212a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:44:07.556550 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1834f6-b19e-4824-bf3e-6a4eda7212a3 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:44:07.697582 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71754b9d-f6f9-4db1-940e-8226bd31a254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:44:07.698420 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71754b9d-f6f9-4db1-940e-8226bd31a254 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:07.698846 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71754b9d-f6f9-4db1-940e-8226bd31a254 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:07.699336 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71754b9d-f6f9-4db1-940e-8226bd31a254 admin admin] PATH_INFO: `/v3/projects/4477f89809db4de8b62f16f570e96f4a` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:07.701864 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71754b9d-f6f9-4db1-940e-8226bd31a254 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4477f89809db4de8b62f16f570e96f4a)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:44:07.704485 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71754b9d-f6f9-4db1-940e-8226bd31a254 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:44:07.869859 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-71754b9d-f6f9-4db1-940e-8226bd31a254 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4477f89809db4de8b62f16f570e96f4a', 'request_id': 'req-71754b9d-f6f9-4db1-940e-8226bd31a254'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:44:07.955070 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-71754b9d-f6f9-4db1-940e-8226bd31a254 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4477f89809db4de8b62f16f570e96f4a', 'request_id': 'req-71754b9d-f6f9-4db1-940e-8226bd31a254'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:44:07.962900 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-71754b9d-f6f9-4db1-940e-8226bd31a254 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4477f89809db4de8b62f16f570e96f4a', 'request_id': 'req-71754b9d-f6f9-4db1-940e-8226bd31a254'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:44:07.990662 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-71754b9d-f6f9-4db1-940e-8226bd31a254 admin admin] The token cache is being invalidate because project 4477f89809db4de8b62f16f570e96f4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 08:44:07.990969 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-71754b9d-f6f9-4db1-940e-8226bd31a254 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:44:07.994262 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-71754b9d-f6f9-4db1-940e-8226bd31a254 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:44:08.000952 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1089/2190] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 08:44:07 2022] DELETE /identity/v3/projects/4477f89809db4de8b62f16f570e96f4a => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:44:08.003807 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d847fc91-d023-4b29-821f-5a54ebc89c7d None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:08.004296 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d847fc91-d023-4b29-821f-5a54ebc89c7d None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:08.004600 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d847fc91-d023-4b29-821f-5a54ebc89c7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:08.055807 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-d847fc91-d023-4b29-821f-5a54ebc89c7d None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:44:08.057261 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-d847fc91-d023-4b29-821f-5a54ebc89c7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:44:08.136969 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1102/2191] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:44:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:44:08.141619 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfd4278-6bdd-435b-a43c-8b116e5aa4a0 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:08.141912 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfd4278-6bdd-435b-a43c-8b116e5aa4a0 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:08.142185 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfd4278-6bdd-435b-a43c-8b116e5aa4a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:08.171916 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-5bfd4278-6bdd-435b-a43c-8b116e5aa4a0 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:44:08.173416 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-5bfd4278-6bdd-435b-a43c-8b116e5aa4a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:44:08.186894 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1090/2192] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:44:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:44:08.191665 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d847fc91-d023-4b29-821f-5a54ebc89c7d None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:44:08.203737 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d847fc91-d023-4b29-821f-5a54ebc89c7d None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:44:08.210648 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4506893-b62b-40bd-b925-ebe9c8b17657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:44:08.211126 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4506893-b62b-40bd-b925-ebe9c8b17657 admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:08.211410 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4506893-b62b-40bd-b925-ebe9c8b17657 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:08.212133 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4506893-b62b-40bd-b925-ebe9c8b17657 admin admin] PATH_INFO: `/v3/domains` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:08.212686 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4506893-b62b-40bd-b925-ebe9c8b17657 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:44:08.213074 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4506893-b62b-40bd-b925-ebe9c8b17657 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:44:08.215516 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4506893-b62b-40bd-b925-ebe9c8b17657 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:44:08.230194 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1103/2193] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Aug 10 08:44:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:44:08.235016 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfd4278-6bdd-435b-a43c-8b116e5aa4a0 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:44:08.246798 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfd4278-6bdd-435b-a43c-8b116e5aa4a0 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:44:08.254597 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277c0739-4985-4c51-8307-e6a323630ec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:44:08.256174 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277c0739-4985-4c51-8307-e6a323630ec1 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:08.256694 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277c0739-4985-4c51-8307-e6a323630ec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:08.257172 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277c0739-4985-4c51-8307-e6a323630ec1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:08.258000 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277c0739-4985-4c51-8307-e6a323630ec1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:44:08.262296 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277c0739-4985-4c51-8307-e6a323630ec1 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:44:08.368306 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4506893-b62b-40bd-b925-ebe9c8b17657 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:44:08.377138 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1091/2194] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 08:44:08 2022] POST /identity/v3/projects => generated 385 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 08:44:08.383979 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4506893-b62b-40bd-b925-ebe9c8b17657 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:44:08.483782 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9cbd08-f226-4c20-816e-16cb561e15ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:44:08.484563 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9cbd08-f226-4c20-816e-16cb561e15ed admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:08.484850 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9cbd08-f226-4c20-816e-16cb561e15ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:08.485836 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9cbd08-f226-4c20-816e-16cb561e15ed admin admin] PATH_INFO: `/v3/users` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:08.486744 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9cbd08-f226-4c20-816e-16cb561e15ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:44:08.489871 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9cbd08-f226-4c20-816e-16cb561e15ed admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:44:08.611513 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1104/2195] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 08:44:08 2022] POST /identity/v3/users => generated 293 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:44:08.615704 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277c0739-4985-4c51-8307-e6a323630ec1 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:44:08.635506 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277c0739-4985-4c51-8307-e6a323630ec1 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:44:08.641145 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4436fe0-d09f-4db1-8dce-2e96de1be63e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:44:08.641929 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4436fe0-d09f-4db1-8dce-2e96de1be63e admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:08.642416 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4436fe0-d09f-4db1-8dce-2e96de1be63e admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:08.642875 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4436fe0-d09f-4db1-8dce-2e96de1be63e admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:08.643585 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4436fe0-d09f-4db1-8dce-2e96de1be63e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:44:08.645225 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4436fe0-d09f-4db1-8dce-2e96de1be63e admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:44:08.664360 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1092/2196] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:44:08 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:44:08.664684 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9cbd08-f226-4c20-816e-16cb561e15ed admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:44:08.678850 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9cbd08-f226-4c20-816e-16cb561e15ed admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:44:08.683909 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e807487d-6b45-4e16-a5c4-077107fa8469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:44:08.685037 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e807487d-6b45-4e16-a5c4-077107fa8469 admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:08.685328 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e807487d-6b45-4e16-a5c4-077107fa8469 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:08.686578 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e807487d-6b45-4e16-a5c4-077107fa8469 admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:08.687322 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e807487d-6b45-4e16-a5c4-077107fa8469 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:44:08.689026 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e807487d-6b45-4e16-a5c4-077107fa8469 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:44:08.707637 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1105/2197] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:44:08 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:44:08.708493 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4436fe0-d09f-4db1-8dce-2e96de1be63e admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:44:08.719662 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4436fe0-d09f-4db1-8dce-2e96de1be63e admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:44:08.731565 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48cbacc-e59e-4dad-b5ae-d796ceacadac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:44:08.731565 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48cbacc-e59e-4dad-b5ae-d796ceacadac admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:08.731565 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48cbacc-e59e-4dad-b5ae-d796ceacadac admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:08.731565 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48cbacc-e59e-4dad-b5ae-d796ceacadac admin admin] PATH_INFO: `/v3/projects/090fe4a1c6a945ba9251a311595ebb82/users/026b89ef6a9a47dbbec34554acc96b5c/roles/44b64db484f24a46bf0b7ced46b519bf` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:08.735437 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e807487d-6b45-4e16-a5c4-077107fa8469 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:44:08.737786 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-e807487d-6b45-4e16-a5c4-077107fa8469 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:44:08.750593 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48cbacc-e59e-4dad-b5ae-d796ceacadac admin admin] RBAC: Authorizing `identity:create_grant(project_id=090fe4a1c6a945ba9251a311595ebb82, user_id=026b89ef6a9a47dbbec34554acc96b5c, role_id=44b64db484f24a46bf0b7ced46b519bf)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:44:08.754608 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48cbacc-e59e-4dad-b5ae-d796ceacadac admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:44:08.759873 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:44:08.770975 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e807487d-6b45-4e16-a5c4-077107fa8469 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:44:08.777092 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3abf42-507a-4785-8bce-41fccba90d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:44:08.777768 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3abf42-507a-4785-8bce-41fccba90d1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:08.778048 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3abf42-507a-4785-8bce-41fccba90d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:08.778319 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3abf42-507a-4785-8bce-41fccba90d1f admin admin] PATH_INFO: `/v3/users/144b9c05a1be449bb7aaf22857088237` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:08.780200 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3abf42-507a-4785-8bce-41fccba90d1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=144b9c05a1be449bb7aaf22857088237)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:44:08.782371 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3abf42-507a-4785-8bce-41fccba90d1f admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:44:08.852629 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1093/2198] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 08:44:08 2022] PUT /identity/v3/projects/090fe4a1c6a945ba9251a311595ebb82/users/026b89ef6a9a47dbbec34554acc96b5c/roles/44b64db484f24a46bf0b7ced46b519bf => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 08:44:08.864273 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d335f1-2fa9-4c42-bc08-f0be319b2a3d None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:08.864825 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d335f1-2fa9-4c42-bc08-f0be319b2a3d None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:08.865327 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d335f1-2fa9-4c42-bc08-f0be319b2a3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:08.900366 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-d5d335f1-2fa9-4c42-bc08-f0be319b2a3d None None] MFA Rules not processed for user `026b89ef6a9a47dbbec34554acc96b5c`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:44:08.901915 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-d5d335f1-2fa9-4c42-bc08-f0be319b2a3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:44:09.007446 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1094/2199] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:44:08 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 08:44:09.016140 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d335f1-2fa9-4c42-bc08-f0be319b2a3d None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:44:09.019470 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-d5d335f1-2fa9-4c42-bc08-f0be319b2a3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:44:09.027605 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-3e3abf42-507a-4785-8bce-41fccba90d1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '144b9c05a1be449bb7aaf22857088237'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:44:09.029539 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:44:09.051156 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-3e3abf42-507a-4785-8bce-41fccba90d1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '144b9c05a1be449bb7aaf22857088237'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:44:09.060835 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-3e3abf42-507a-4785-8bce-41fccba90d1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '144b9c05a1be449bb7aaf22857088237'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:44:09.141382 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-3e3abf42-507a-4785-8bce-41fccba90d1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '144b9c05a1be449bb7aaf22857088237'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:44:09.142147 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d335f1-2fa9-4c42-bc08-f0be319b2a3d None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:44:09.142325 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-3e3abf42-507a-4785-8bce-41fccba90d1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '144b9c05a1be449bb7aaf22857088237'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:44:09.146223 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10c14e7-8e45-410e-82b9-52886b50c95f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ba74e28ced4cbba857f54222dfbe41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:44:09.146910 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10c14e7-8e45-410e-82b9-52886b50c95f service cinder] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:09.147234 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10c14e7-8e45-410e-82b9-52886b50c95f service cinder] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:09.148064 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10c14e7-8e45-410e-82b9-52886b50c95f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:09.149480 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-d10c14e7-8e45-410e-82b9-52886b50c95f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:44:09.152125 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1106/2200] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 08:44:08 2022] DELETE /identity/v3/users/144b9c05a1be449bb7aaf22857088237 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:44:09.158015 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443bc6d1-156a-4525-850a-05cef919ff92 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:09.158351 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443bc6d1-156a-4525-850a-05cef919ff92 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:09.158736 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443bc6d1-156a-4525-850a-05cef919ff92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:09.166590 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10c14e7-8e45-410e-82b9-52886b50c95f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:44:09.168594 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10c14e7-8e45-410e-82b9-52886b50c95f service cinder] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:44:09.184232 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1095/2201] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:44:09 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:44:09.198034 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-443bc6d1-156a-4525-850a-05cef919ff92 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:44:09.209649 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-443bc6d1-156a-4525-850a-05cef919ff92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:44:09.213218 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10c14e7-8e45-410e-82b9-52886b50c95f service cinder] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:44:09.218044 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-d10c14e7-8e45-410e-82b9-52886b50c95f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:44:09.314376 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1107/2202] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:44:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:44:09.315550 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10c14e7-8e45-410e-82b9-52886b50c95f service cinder] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:44:09.323592 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443bc6d1-156a-4525-850a-05cef919ff92 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:44:09.324405 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6282a7ed-a543-4c7e-a5b7-35222539b36f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:44:09.325235 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6282a7ed-a543-4c7e-a5b7-35222539b36f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:09.325512 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6282a7ed-a543-4c7e-a5b7-35222539b36f admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:09.325780 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6282a7ed-a543-4c7e-a5b7-35222539b36f admin admin] PATH_INFO: `/v3/users/026b89ef6a9a47dbbec34554acc96b5c` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:09.325897 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-443bc6d1-156a-4525-850a-05cef919ff92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:44:09.329124 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6282a7ed-a543-4c7e-a5b7-35222539b36f admin admin] RBAC: Authorizing `identity:delete_user(user_id=026b89ef6a9a47dbbec34554acc96b5c)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:44:09.331118 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6282a7ed-a543-4c7e-a5b7-35222539b36f admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:44:09.411273 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443bc6d1-156a-4525-850a-05cef919ff92 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:44:09.416652 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1138e02-2860-428c-ab15-377006a27d6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:44:09.417770 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1138e02-2860-428c-ab15-377006a27d6e service neutron] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:09.418469 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1138e02-2860-428c-ab15-377006a27d6e service neutron] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:09.418469 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1138e02-2860-428c-ab15-377006a27d6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:09.431979 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1138e02-2860-428c-ab15-377006a27d6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:44:09.433626 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1138e02-2860-428c-ab15-377006a27d6e service neutron] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:44:09.445897 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1108/2203] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:44:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 08:44:09.614284 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:44:09.636423 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-6282a7ed-a543-4c7e-a5b7-35222539b36f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '026b89ef6a9a47dbbec34554acc96b5c'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:44:09.658360 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-6282a7ed-a543-4c7e-a5b7-35222539b36f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '026b89ef6a9a47dbbec34554acc96b5c'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:44:09.691732 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-6282a7ed-a543-4c7e-a5b7-35222539b36f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '026b89ef6a9a47dbbec34554acc96b5c'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:44:09.693257 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-6282a7ed-a543-4c7e-a5b7-35222539b36f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '026b89ef6a9a47dbbec34554acc96b5c'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:44:09.694215 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-6282a7ed-a543-4c7e-a5b7-35222539b36f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '026b89ef6a9a47dbbec34554acc96b5c'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:44:09.787647 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1096/2204] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 08:44:09 2022] DELETE /identity/v3/users/026b89ef6a9a47dbbec34554acc96b5c => generated 0 bytes in 576 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:44:09.789711 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80fe8c8-563f-4e8f-800d-829ef9b50b27 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:09.790003 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80fe8c8-563f-4e8f-800d-829ef9b50b27 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:09.790279 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80fe8c8-563f-4e8f-800d-829ef9b50b27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:09.821513 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-a80fe8c8-563f-4e8f-800d-829ef9b50b27 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:44:09.822469 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-a80fe8c8-563f-4e8f-800d-829ef9b50b27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:44:09.939939 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1109/2205] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:44:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:44:09.953113 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6282a7ed-a543-4c7e-a5b7-35222539b36f admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:44:09.954951 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-6282a7ed-a543-4c7e-a5b7-35222539b36f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:44:10.016236 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6282a7ed-a543-4c7e-a5b7-35222539b36f admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:44:10.021575 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d6d972-8054-4662-b55b-a5a4f0809035 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:44:10.024015 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d6d972-8054-4662-b55b-a5a4f0809035 service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:10.024015 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d6d972-8054-4662-b55b-a5a4f0809035 service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:10.024015 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d6d972-8054-4662-b55b-a5a4f0809035 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:10.033745 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d6d972-8054-4662-b55b-a5a4f0809035 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:44:10.035197 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d6d972-8054-4662-b55b-a5a4f0809035 service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:44:10.053831 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80fe8c8-563f-4e8f-800d-829ef9b50b27 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:44:10.057853 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1097/2206] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:44:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:44:10.058421 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-a80fe8c8-563f-4e8f-800d-829ef9b50b27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:44:10.079848 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80fe8c8-563f-4e8f-800d-829ef9b50b27 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:44:10.095893 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e603244-ac6a-4acb-a83f-a6312ebf7683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:44:10.096619 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e603244-ac6a-4acb-a83f-a6312ebf7683 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:10.096906 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e603244-ac6a-4acb-a83f-a6312ebf7683 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:10.097349 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e603244-ac6a-4acb-a83f-a6312ebf7683 admin admin] PATH_INFO: `/v3/projects/3cf74f95d149492ba14ed43020bf3f6e` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:10.101764 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e603244-ac6a-4acb-a83f-a6312ebf7683 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3cf74f95d149492ba14ed43020bf3f6e)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:44:10.106636 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e603244-ac6a-4acb-a83f-a6312ebf7683 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:44:10.110501 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:44:10.259772 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-3e603244-ac6a-4acb-a83f-a6312ebf7683 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3cf74f95d149492ba14ed43020bf3f6e', 'request_id': 'req-3e603244-ac6a-4acb-a83f-a6312ebf7683'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:44:10.271802 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-3e603244-ac6a-4acb-a83f-a6312ebf7683 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3cf74f95d149492ba14ed43020bf3f6e', 'request_id': 'req-3e603244-ac6a-4acb-a83f-a6312ebf7683'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:44:10.391868 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-3e603244-ac6a-4acb-a83f-a6312ebf7683 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3cf74f95d149492ba14ed43020bf3f6e', 'request_id': 'req-3e603244-ac6a-4acb-a83f-a6312ebf7683'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:44:10.412717 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-3e603244-ac6a-4acb-a83f-a6312ebf7683 admin admin] The token cache is being invalidate because project 3cf74f95d149492ba14ed43020bf3f6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 08:44:10.413270 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-3e603244-ac6a-4acb-a83f-a6312ebf7683 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:44:10.414045 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-3e603244-ac6a-4acb-a83f-a6312ebf7683 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:44:10.416553 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1110/2207] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 08:44:10 2022] DELETE /identity/v3/projects/3cf74f95d149492ba14ed43020bf3f6e => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 08:44:10.423800 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a1d949-c8c4-46be-9f35-55fe2fcfdeb6 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:10.424088 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a1d949-c8c4-46be-9f35-55fe2fcfdeb6 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:10.424364 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a1d949-c8c4-46be-9f35-55fe2fcfdeb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:10.451974 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-e5a1d949-c8c4-46be-9f35-55fe2fcfdeb6 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:44:10.453331 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-e5a1d949-c8c4-46be-9f35-55fe2fcfdeb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:44:10.569489 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1098/2208] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:44:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 08:44:10.572850 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22cc65f-203b-4c2b-9e73-787362ba826e None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:10.573194 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22cc65f-203b-4c2b-9e73-787362ba826e None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:10.573476 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22cc65f-203b-4c2b-9e73-787362ba826e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:10.596624 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-d22cc65f-203b-4c2b-9e73-787362ba826e None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:44:10.598323 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-d22cc65f-203b-4c2b-9e73-787362ba826e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:44:10.609771 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1111/2209] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:44:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 08:44:10.614042 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a1d949-c8c4-46be-9f35-55fe2fcfdeb6 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:44:10.626633 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a1d949-c8c4-46be-9f35-55fe2fcfdeb6 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:44:10.631753 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0f91c3-4bb0-4e20-b86b-2f3d21817605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:44:10.632465 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0f91c3-4bb0-4e20-b86b-2f3d21817605 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:10.632869 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0f91c3-4bb0-4e20-b86b-2f3d21817605 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:10.633180 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0f91c3-4bb0-4e20-b86b-2f3d21817605 admin admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:10.633986 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0f91c3-4bb0-4e20-b86b-2f3d21817605 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:44:10.634292 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0f91c3-4bb0-4e20-b86b-2f3d21817605 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:44:10.635912 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0f91c3-4bb0-4e20-b86b-2f3d21817605 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:44:10.650203 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1099/2210] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Aug 10 08:44:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:44:10.655155 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22cc65f-203b-4c2b-9e73-787362ba826e None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:44:10.665723 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22cc65f-203b-4c2b-9e73-787362ba826e None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:44:10.671414 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56f64a6-0e3e-4a47-b4a2-ac38e06071c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:44:10.672021 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56f64a6-0e3e-4a47-b4a2-ac38e06071c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:10.672305 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56f64a6-0e3e-4a47-b4a2-ac38e06071c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:10.672609 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56f64a6-0e3e-4a47-b4a2-ac38e06071c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:10.673300 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56f64a6-0e3e-4a47-b4a2-ac38e06071c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:44:10.675305 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56f64a6-0e3e-4a47-b4a2-ac38e06071c1 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:44:10.776292 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0f91c3-4bb0-4e20-b86b-2f3d21817605 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:44:10.783688 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-9c0f91c3-4bb0-4e20-b86b-2f3d21817605 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:44:10.793875 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1112/2211] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 08:44:10 2022] POST /identity/v3/projects => generated 377 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:44:10.794850 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56f64a6-0e3e-4a47-b4a2-ac38e06071c1 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:44:10.807253 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0f91c3-4bb0-4e20-b86b-2f3d21817605 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:44:11.052515 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:44:11.056341 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56f64a6-0e3e-4a47-b4a2-ac38e06071c1 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:44:11.057642 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e468a464-e205-48bf-a8c1-821b1d2a0fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:44:11.057642 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e468a464-e205-48bf-a8c1-821b1d2a0fb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:11.057642 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e468a464-e205-48bf-a8c1-821b1d2a0fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:11.057642 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e468a464-e205-48bf-a8c1-821b1d2a0fb1 admin admin] PATH_INFO: `/v3/projects/090fe4a1c6a945ba9251a311595ebb82` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:11.058092 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e468a464-e205-48bf-a8c1-821b1d2a0fb1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=090fe4a1c6a945ba9251a311595ebb82)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:44:11.058092 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e468a464-e205-48bf-a8c1-821b1d2a0fb1 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:44:11.058266 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e908ab-ee8c-487b-8d21-2d367d546914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:44:11.058266 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e908ab-ee8c-487b-8d21-2d367d546914 admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:11.058266 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e908ab-ee8c-487b-8d21-2d367d546914 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:11.058266 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e908ab-ee8c-487b-8d21-2d367d546914 admin admin] PATH_INFO: `/v3/users` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:11.058606 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e908ab-ee8c-487b-8d21-2d367d546914 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:44:11.058606 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e908ab-ee8c-487b-8d21-2d367d546914 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:44:11.187384 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1113/2212] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 08:44:10 2022] POST /identity/v3/users => generated 289 bytes in 391 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:44:11.187384 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27040ea2-24d3-4030-b6c5-c9db4c8106ad None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:11.190146 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27040ea2-24d3-4030-b6c5-c9db4c8106ad None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:11.190489 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27040ea2-24d3-4030-b6c5-c9db4c8106ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:11.204770 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-e468a464-e205-48bf-a8c1-821b1d2a0fb1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '090fe4a1c6a945ba9251a311595ebb82', 'request_id': 'req-e468a464-e205-48bf-a8c1-821b1d2a0fb1'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:44:11.224701 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-27040ea2-24d3-4030-b6c5-c9db4c8106ad None None] MFA Rules not processed for user `f553fa4ae3ca41d68d0872e13d8161cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:44:11.229696 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-27040ea2-24d3-4030-b6c5-c9db4c8106ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:44:11.274844 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:44:11.291703 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-e468a464-e205-48bf-a8c1-821b1d2a0fb1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '090fe4a1c6a945ba9251a311595ebb82', 'request_id': 'req-e468a464-e205-48bf-a8c1-821b1d2a0fb1'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:44:11.298859 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-e468a464-e205-48bf-a8c1-821b1d2a0fb1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '090fe4a1c6a945ba9251a311595ebb82', 'request_id': 'req-e468a464-e205-48bf-a8c1-821b1d2a0fb1'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:44:11.304341 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1114/2213] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:44:11 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:44:11.305299 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27040ea2-24d3-4030-b6c5-c9db4c8106ad None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:44:11.311473 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-e468a464-e205-48bf-a8c1-821b1d2a0fb1 admin admin] The token cache is being invalidate because project 090fe4a1c6a945ba9251a311595ebb82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 08:44:11.311790 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-e468a464-e205-48bf-a8c1-821b1d2a0fb1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:44:11.312444 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-e468a464-e205-48bf-a8c1-821b1d2a0fb1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:44:11.313989 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1100/2214] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 08:44:10 2022] DELETE /identity/v3/projects/090fe4a1c6a945ba9251a311595ebb82 => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:44:11.318805 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e4a17e-639d-4b0b-8509-b66be2dc2fe5 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:11.323793 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e4a17e-639d-4b0b-8509-b66be2dc2fe5 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:11.323793 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e4a17e-639d-4b0b-8509-b66be2dc2fe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:11.342092 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-49e4a17e-639d-4b0b-8509-b66be2dc2fe5 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:44:11.343610 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-49e4a17e-639d-4b0b-8509-b66be2dc2fe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:44:11.383677 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27040ea2-24d3-4030-b6c5-c9db4c8106ad None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:44:11.393690 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db16dfc7-b487-4bef-8d95-b1cb64b49667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:44:11.395249 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db16dfc7-b487-4bef-8d95-b1cb64b49667 admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:11.395534 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db16dfc7-b487-4bef-8d95-b1cb64b49667 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:11.395809 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db16dfc7-b487-4bef-8d95-b1cb64b49667 admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:11.396503 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db16dfc7-b487-4bef-8d95-b1cb64b49667 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:44:11.398385 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db16dfc7-b487-4bef-8d95-b1cb64b49667 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:44:11.414841 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1115/2215] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:44:11 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:44:11.417958 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db16dfc7-b487-4bef-8d95-b1cb64b49667 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:44:11.417958 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-db16dfc7-b487-4bef-8d95-b1cb64b49667 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:44:11.421791 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1101/2216] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:44:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:44:11.422561 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e4a17e-639d-4b0b-8509-b66be2dc2fe5 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:44:11.423863 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-49e4a17e-639d-4b0b-8509-b66be2dc2fe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:44:11.436368 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e4a17e-639d-4b0b-8509-b66be2dc2fe5 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:44:11.441994 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cec315e-01c4-4edf-92a2-faf8c60d6f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:44:11.442606 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cec315e-01c4-4edf-92a2-faf8c60d6f30 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:11.443187 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cec315e-01c4-4edf-92a2-faf8c60d6f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:11.443187 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cec315e-01c4-4edf-92a2-faf8c60d6f30 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:11.443788 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cec315e-01c4-4edf-92a2-faf8c60d6f30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:44:11.445487 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cec315e-01c4-4edf-92a2-faf8c60d6f30 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:44:11.456176 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1102/2217] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:44:11 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:44:11.457498 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa75925-90ad-428b-b248-36abde52c62a None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:11.457776 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa75925-90ad-428b-b248-36abde52c62a None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:11.458054 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa75925-90ad-428b-b248-36abde52c62a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:11.481498 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-9fa75925-90ad-428b-b248-36abde52c62a None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:44:11.483287 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-9fa75925-90ad-428b-b248-36abde52c62a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:44:11.497169 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1103/2218] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:44:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:44:11.499495 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa75925-90ad-428b-b248-36abde52c62a None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:44:11.511310 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa75925-90ad-428b-b248-36abde52c62a None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:44:11.514471 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db16dfc7-b487-4bef-8d95-b1cb64b49667 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:44:11.519748 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1641c6a0-7dcb-43ef-9378-0ae457bf615a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:44:11.519748 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1641c6a0-7dcb-43ef-9378-0ae457bf615a admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:11.519748 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1641c6a0-7dcb-43ef-9378-0ae457bf615a admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:11.519748 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1641c6a0-7dcb-43ef-9378-0ae457bf615a admin admin] PATH_INFO: `/v3/projects/d1f83b4ed9ee42df9f946c9b4e13708d/users/956ccf8932b24a7f8d74f4cbc3ead23a/roles/44b64db484f24a46bf0b7ced46b519bf` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:11.520788 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae44597-609f-4bdf-89e2-37e771c98a3f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a943008193914f76a663d489bd73e873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:44:11.521506 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae44597-609f-4bdf-89e2-37e771c98a3f service nova] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:11.521860 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae44597-609f-4bdf-89e2-37e771c98a3f service nova] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:11.522206 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae44597-609f-4bdf-89e2-37e771c98a3f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:11.524112 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-dae44597-609f-4bdf-89e2-37e771c98a3f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:44:11.535269 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1641c6a0-7dcb-43ef-9378-0ae457bf615a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1f83b4ed9ee42df9f946c9b4e13708d, user_id=956ccf8932b24a7f8d74f4cbc3ead23a, role_id=44b64db484f24a46bf0b7ced46b519bf)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:44:11.538114 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1641c6a0-7dcb-43ef-9378-0ae457bf615a admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:44:11.588565 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae44597-609f-4bdf-89e2-37e771c98a3f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:44:11.590289 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae44597-609f-4bdf-89e2-37e771c98a3f service nova] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:44:11.598459 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1104/2219] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 08:44:11 2022] PUT /identity/v3/projects/d1f83b4ed9ee42df9f946c9b4e13708d/users/956ccf8932b24a7f8d74f4cbc3ead23a/roles/44b64db484f24a46bf0b7ced46b519bf => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:44:11.598459 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1641c6a0-7dcb-43ef-9378-0ae457bf615a admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:44:11.613847 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1116/2220] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:44:11 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:44:11.615323 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcf59e4-324c-4500-a039-88da30978135 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:11.615613 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcf59e4-324c-4500-a039-88da30978135 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:11.615883 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcf59e4-324c-4500-a039-88da30978135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:11.642709 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-1dcf59e4-324c-4500-a039-88da30978135 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:44:11.647792 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-1dcf59e4-324c-4500-a039-88da30978135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:44:11.649616 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:44:11.663630 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:44:11.704852 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1641c6a0-7dcb-43ef-9378-0ae457bf615a admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:44:11.710823 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8285849a-4b12-48e6-b8e4-5e929f842f21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:44:11.715080 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8285849a-4b12-48e6-b8e4-5e929f842f21 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:11.715080 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8285849a-4b12-48e6-b8e4-5e929f842f21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:11.715080 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8285849a-4b12-48e6-b8e4-5e929f842f21 admin admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:11.715080 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8285849a-4b12-48e6-b8e4-5e929f842f21 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:44:11.715080 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8285849a-4b12-48e6-b8e4-5e929f842f21 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:44:11.716514 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8285849a-4b12-48e6-b8e4-5e929f842f21 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:44:11.730051 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1105/2221] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Aug 10 08:44:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:44:11.735386 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8285849a-4b12-48e6-b8e4-5e929f842f21 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:44:11.740815 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1117/2222] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:44:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:44:11.746478 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8285849a-4b12-48e6-b8e4-5e929f842f21 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:44:11.747594 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcf59e4-324c-4500-a039-88da30978135 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:44:11.749893 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-1dcf59e4-324c-4500-a039-88da30978135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:44:11.752948 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0e59bf-00e7-4bee-9ec5-9fd15c8bf417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:44:11.753483 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0e59bf-00e7-4bee-9ec5-9fd15c8bf417 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:11.753768 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0e59bf-00e7-4bee-9ec5-9fd15c8bf417 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:11.754041 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0e59bf-00e7-4bee-9ec5-9fd15c8bf417 admin admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:11.754841 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0e59bf-00e7-4bee-9ec5-9fd15c8bf417 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:44:11.757015 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0e59bf-00e7-4bee-9ec5-9fd15c8bf417 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:44:11.831126 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcf59e4-324c-4500-a039-88da30978135 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:44:11.835239 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928ffade-f0b9-49f4-a960-98f9c6a0e59e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:44:11.835855 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928ffade-f0b9-49f4-a960-98f9c6a0e59e service neutron] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:11.836137 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928ffade-f0b9-49f4-a960-98f9c6a0e59e service neutron] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:11.836409 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928ffade-f0b9-49f4-a960-98f9c6a0e59e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:11.847584 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928ffade-f0b9-49f4-a960-98f9c6a0e59e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:44:11.851190 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928ffade-f0b9-49f4-a960-98f9c6a0e59e service neutron] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:44:11.859296 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1118/2223] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:44:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:44:11.878295 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1106/2224] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 08:44:11 2022] POST /identity/v3/projects => generated 385 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:44:11.882615 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928ffade-f0b9-49f4-a960-98f9c6a0e59e service neutron] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:44:11.893184 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928ffade-f0b9-49f4-a960-98f9c6a0e59e service neutron] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:44:11.963317 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b17b23-7f19-4b4f-a865-7f414cbe15ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:44:11.963934 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b17b23-7f19-4b4f-a865-7f414cbe15ac admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:11.964334 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b17b23-7f19-4b4f-a865-7f414cbe15ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:11.964611 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b17b23-7f19-4b4f-a865-7f414cbe15ac admin admin] PATH_INFO: `/v3/users` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:11.965300 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b17b23-7f19-4b4f-a865-7f414cbe15ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:44:11.967804 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b17b23-7f19-4b4f-a865-7f414cbe15ac admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:44:12.048194 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1119/2225] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 08:44:11 2022] POST /identity/v3/users => generated 293 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:44:12.052508 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0e59bf-00e7-4bee-9ec5-9fd15c8bf417 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:44:12.064490 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0e59bf-00e7-4bee-9ec5-9fd15c8bf417 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:44:12.070737 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cdd6dc-6f94-4135-a200-9759e313e606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:44:12.071209 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cdd6dc-6f94-4135-a200-9759e313e606 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:12.071482 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cdd6dc-6f94-4135-a200-9759e313e606 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:12.071708 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cdd6dc-6f94-4135-a200-9759e313e606 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:12.072424 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cdd6dc-6f94-4135-a200-9759e313e606 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:44:12.074077 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cdd6dc-6f94-4135-a200-9759e313e606 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:44:12.085285 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1107/2226] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:44:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:44:12.089285 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b17b23-7f19-4b4f-a865-7f414cbe15ac admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:44:12.099843 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b17b23-7f19-4b4f-a865-7f414cbe15ac admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:44:12.106558 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72b6c88-dab5-4f45-bc23-547e37f4db61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:44:12.106718 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72b6c88-dab5-4f45-bc23-547e37f4db61 admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:12.106909 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72b6c88-dab5-4f45-bc23-547e37f4db61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:12.107182 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72b6c88-dab5-4f45-bc23-547e37f4db61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:12.108372 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72b6c88-dab5-4f45-bc23-547e37f4db61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:44:12.110063 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72b6c88-dab5-4f45-bc23-547e37f4db61 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:44:12.120433 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1120/2227] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:44:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:44:12.125473 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cdd6dc-6f94-4135-a200-9759e313e606 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:44:12.135982 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cdd6dc-6f94-4135-a200-9759e313e606 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:44:12.140743 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c21833-0e4d-4f9d-80ad-1dfcfcf834b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:44:12.141526 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c21833-0e4d-4f9d-80ad-1dfcfcf834b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:12.141812 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c21833-0e4d-4f9d-80ad-1dfcfcf834b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:12.142087 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c21833-0e4d-4f9d-80ad-1dfcfcf834b4 admin admin] PATH_INFO: `/v3/projects/c44b7423b80d45c8b04f6df370115171/users/a1a172cfb48f42b295d9ff34f3d72598/roles/44b64db484f24a46bf0b7ced46b519bf` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:12.158283 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c21833-0e4d-4f9d-80ad-1dfcfcf834b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c44b7423b80d45c8b04f6df370115171, user_id=a1a172cfb48f42b295d9ff34f3d72598, role_id=44b64db484f24a46bf0b7ced46b519bf)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:44:12.161054 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c21833-0e4d-4f9d-80ad-1dfcfcf834b4 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:44:12.279459 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1108/2228] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 08:44:12 2022] PUT /identity/v3/projects/c44b7423b80d45c8b04f6df370115171/users/a1a172cfb48f42b295d9ff34f3d72598/roles/44b64db484f24a46bf0b7ced46b519bf => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:44:12.281255 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7931a610-e2b2-4681-9b67-117a508b497a None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:12.281544 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7931a610-e2b2-4681-9b67-117a508b497a None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:12.281817 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7931a610-e2b2-4681-9b67-117a508b497a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:12.314702 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-7931a610-e2b2-4681-9b67-117a508b497a None None] MFA Rules not processed for user `a1a172cfb48f42b295d9ff34f3d72598`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:44:12.318479 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-7931a610-e2b2-4681-9b67-117a508b497a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:44:12.341728 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:44:12.390663 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1121/2229] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:44:12 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 08:44:12.407503 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c21833-0e4d-4f9d-80ad-1dfcfcf834b4 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:44:12.523538 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c21833-0e4d-4f9d-80ad-1dfcfcf834b4 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:44:12.528586 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5f989c-8e5a-47b4-9697-697e04c4e4fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:44:12.530648 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5f989c-8e5a-47b4-9697-697e04c4e4fa admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:12.530942 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5f989c-8e5a-47b4-9697-697e04c4e4fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:12.531355 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5f989c-8e5a-47b4-9697-697e04c4e4fa admin admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:12.532019 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5f989c-8e5a-47b4-9697-697e04c4e4fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:44:12.534298 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5f989c-8e5a-47b4-9697-697e04c4e4fa admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:44:12.662324 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1109/2230] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 08:44:12 2022] POST /identity/v3/projects => generated 383 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 08:44:12.663489 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7931a610-e2b2-4681-9b67-117a508b497a None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:44:12.674193 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7931a610-e2b2-4681-9b67-117a508b497a None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:44:12.730056 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f5fd0d-5a1f-4360-8d88-a5f5d7a4f311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:44:12.730694 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f5fd0d-5a1f-4360-8d88-a5f5d7a4f311 admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:12.730977 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f5fd0d-5a1f-4360-8d88-a5f5d7a4f311 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:12.731250 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f5fd0d-5a1f-4360-8d88-a5f5d7a4f311 admin admin] PATH_INFO: `/v3/users` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:12.731885 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f5fd0d-5a1f-4360-8d88-a5f5d7a4f311 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:44:12.734010 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f5fd0d-5a1f-4360-8d88-a5f5d7a4f311 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:44:12.809322 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1122/2231] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 08:44:12 2022] POST /identity/v3/users => generated 292 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 08:44:12.810470 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5f989c-8e5a-47b4-9697-697e04c4e4fa admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:44:12.822543 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5f989c-8e5a-47b4-9697-697e04c4e4fa admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:44:12.828366 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a27d42-9acb-4568-8a93-705d79be877d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:44:12.829099 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a27d42-9acb-4568-8a93-705d79be877d admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:12.829838 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a27d42-9acb-4568-8a93-705d79be877d admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:12.830335 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a27d42-9acb-4568-8a93-705d79be877d admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:12.831230 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a27d42-9acb-4568-8a93-705d79be877d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:44:12.833668 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a27d42-9acb-4568-8a93-705d79be877d admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:44:12.848958 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1110/2232] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:44:12 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:44:12.852082 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f5fd0d-5a1f-4360-8d88-a5f5d7a4f311 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:44:12.866100 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f5fd0d-5a1f-4360-8d88-a5f5d7a4f311 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:44:12.869383 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0b4a96-fff3-4e2d-8645-7f32626737af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:44:12.870082 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0b4a96-fff3-4e2d-8645-7f32626737af admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:12.870433 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0b4a96-fff3-4e2d-8645-7f32626737af admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:12.870909 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0b4a96-fff3-4e2d-8645-7f32626737af admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:12.871621 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0b4a96-fff3-4e2d-8645-7f32626737af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:44:12.873609 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0b4a96-fff3-4e2d-8645-7f32626737af admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:44:12.888380 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a27d42-9acb-4568-8a93-705d79be877d admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:44:12.889362 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1123/2233] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:44:12 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:44:12.899230 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a27d42-9acb-4568-8a93-705d79be877d admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:44:12.904146 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091576b3-bae1-4915-bf94-57d15e3a2523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:44:12.905422 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091576b3-bae1-4915-bf94-57d15e3a2523 admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:12.905708 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091576b3-bae1-4915-bf94-57d15e3a2523 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:12.905986 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091576b3-bae1-4915-bf94-57d15e3a2523 admin admin] PATH_INFO: `/v3/projects/04070adc4317469794b066ccb48004bf/users/a7fdad6e74a64e44be962f62973b284d/roles/44b64db484f24a46bf0b7ced46b519bf` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:12.924375 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091576b3-bae1-4915-bf94-57d15e3a2523 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04070adc4317469794b066ccb48004bf, user_id=a7fdad6e74a64e44be962f62973b284d, role_id=44b64db484f24a46bf0b7ced46b519bf)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:44:12.927262 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091576b3-bae1-4915-bf94-57d15e3a2523 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:44:13.040956 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1111/2234] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 08:44:12 2022] PUT /identity/v3/projects/04070adc4317469794b066ccb48004bf/users/a7fdad6e74a64e44be962f62973b284d/roles/44b64db484f24a46bf0b7ced46b519bf => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:44:13.043594 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3c084e-9156-41e4-b7a9-7ecdf011c9ab None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:13.044129 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3c084e-9156-41e4-b7a9-7ecdf011c9ab None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:13.044895 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3c084e-9156-41e4-b7a9-7ecdf011c9ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:13.110296 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-ed3c084e-9156-41e4-b7a9-7ecdf011c9ab None None] MFA Rules not processed for user `a7fdad6e74a64e44be962f62973b284d`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:44:13.112512 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-ed3c084e-9156-41e4-b7a9-7ecdf011c9ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:44:13.184940 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1124/2235] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:44:13 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:44:13.190301 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091576b3-bae1-4915-bf94-57d15e3a2523 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:44:13.208771 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:44:13.290286 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091576b3-bae1-4915-bf94-57d15e3a2523 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:44:13.298685 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca31a18-bcb1-4975-b07e-2a6b860ce259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:44:13.299334 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca31a18-bcb1-4975-b07e-2a6b860ce259 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:13.299614 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca31a18-bcb1-4975-b07e-2a6b860ce259 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:13.299883 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca31a18-bcb1-4975-b07e-2a6b860ce259 admin admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:13.300527 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca31a18-bcb1-4975-b07e-2a6b860ce259 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:44:13.302484 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca31a18-bcb1-4975-b07e-2a6b860ce259 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:44:13.413044 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1112/2236] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 08:44:13 2022] POST /identity/v3/projects => generated 385 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:44:13.415191 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3c084e-9156-41e4-b7a9-7ecdf011c9ab None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:44:13.428884 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3c084e-9156-41e4-b7a9-7ecdf011c9ab None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:44:13.473898 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:44:13.527958 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedacf17-a91e-4e35-b607-169f7ea84191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:44:13.528119 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedacf17-a91e-4e35-b607-169f7ea84191 admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:13.528508 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedacf17-a91e-4e35-b607-169f7ea84191 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:13.529465 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedacf17-a91e-4e35-b607-169f7ea84191 admin admin] PATH_INFO: `/v3/users` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:13.531027 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedacf17-a91e-4e35-b607-169f7ea84191 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:44:13.534113 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedacf17-a91e-4e35-b607-169f7ea84191 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:44:13.653778 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1125/2237] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 08:44:13 2022] POST /identity/v3/users => generated 299 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:44:13.658410 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca31a18-bcb1-4975-b07e-2a6b860ce259 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:44:13.678363 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca31a18-bcb1-4975-b07e-2a6b860ce259 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:44:13.693616 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d994959-0ca1-4b0c-8031-5a9bd367145c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:44:13.694383 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d994959-0ca1-4b0c-8031-5a9bd367145c admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:13.694766 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d994959-0ca1-4b0c-8031-5a9bd367145c admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:13.695392 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d994959-0ca1-4b0c-8031-5a9bd367145c admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:13.696277 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d994959-0ca1-4b0c-8031-5a9bd367145c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:44:13.697995 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d994959-0ca1-4b0c-8031-5a9bd367145c admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:44:13.727871 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1113/2238] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:44:13 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:44:13.731794 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedacf17-a91e-4e35-b607-169f7ea84191 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:44:13.754587 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedacf17-a91e-4e35-b607-169f7ea84191 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:44:13.768748 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63711254-45d2-4e87-9ae8-603ca68ffb59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:44:13.769477 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63711254-45d2-4e87-9ae8-603ca68ffb59 admin admin] REQUEST_METHOD: `PUT` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:13.769885 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63711254-45d2-4e87-9ae8-603ca68ffb59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:13.770165 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63711254-45d2-4e87-9ae8-603ca68ffb59 admin admin] PATH_INFO: `/v3/projects/6bb6beb4c596446bb28b7941fe8fd785/users/1833ddad656c4e4ba2dabbaa36eb6b25/roles/ec3751692d9c4ecf81b4a71c936c46be` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:13.802232 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63711254-45d2-4e87-9ae8-603ca68ffb59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bb6beb4c596446bb28b7941fe8fd785, user_id=1833ddad656c4e4ba2dabbaa36eb6b25, role_id=ec3751692d9c4ecf81b4a71c936c46be)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:44:13.808816 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63711254-45d2-4e87-9ae8-603ca68ffb59 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:44:13.925809 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1126/2239] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 08:44:13 2022] PUT /identity/v3/projects/6bb6beb4c596446bb28b7941fe8fd785/users/1833ddad656c4e4ba2dabbaa36eb6b25/roles/ec3751692d9c4ecf81b4a71c936c46be => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:44:13.929746 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d994959-0ca1-4b0c-8031-5a9bd367145c admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:44:13.945834 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d994959-0ca1-4b0c-8031-5a9bd367145c admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:44:14.038586 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec90cbe7-84de-4e55-a1ea-3e9ac7132b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:44:14.039298 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec90cbe7-84de-4e55-a1ea-3e9ac7132b2b admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:14.039578 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec90cbe7-84de-4e55-a1ea-3e9ac7132b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:14.041222 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec90cbe7-84de-4e55-a1ea-3e9ac7132b2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:14.042371 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec90cbe7-84de-4e55-a1ea-3e9ac7132b2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:44:14.047801 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec90cbe7-84de-4e55-a1ea-3e9ac7132b2b admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:44:14.072559 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1114/2240] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:44:13 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:44:14.078616 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63711254-45d2-4e87-9ae8-603ca68ffb59 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:44:14.097656 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63711254-45d2-4e87-9ae8-603ca68ffb59 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:44:14.108048 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e45f521-ded7-427e-bb8e-6a44b9cf589d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:44:14.108868 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e45f521-ded7-427e-bb8e-6a44b9cf589d admin admin] REQUEST_METHOD: `PUT` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:14.109202 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e45f521-ded7-427e-bb8e-6a44b9cf589d admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:14.109499 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e45f521-ded7-427e-bb8e-6a44b9cf589d admin admin] PATH_INFO: `/v3/projects/6bb6beb4c596446bb28b7941fe8fd785/users/1833ddad656c4e4ba2dabbaa36eb6b25/roles/ec3751692d9c4ecf81b4a71c936c46be` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:14.113861 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e45f521-ded7-427e-bb8e-6a44b9cf589d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bb6beb4c596446bb28b7941fe8fd785, user_id=1833ddad656c4e4ba2dabbaa36eb6b25, role_id=ec3751692d9c4ecf81b4a71c936c46be)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:44:14.116537 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e45f521-ded7-427e-bb8e-6a44b9cf589d admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:44:14.143891 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1127/2241] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 08:44:14 2022] PUT /identity/v3/projects/6bb6beb4c596446bb28b7941fe8fd785/users/1833ddad656c4e4ba2dabbaa36eb6b25/roles/ec3751692d9c4ecf81b4a71c936c46be => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:44:14.155798 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854ca2bb-25d5-4c8e-8af3-c2a9c4fda256 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:14.156117 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854ca2bb-25d5-4c8e-8af3-c2a9c4fda256 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:14.156391 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854ca2bb-25d5-4c8e-8af3-c2a9c4fda256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:14.191583 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-854ca2bb-25d5-4c8e-8af3-c2a9c4fda256 None None] MFA Rules not processed for user `1833ddad656c4e4ba2dabbaa36eb6b25`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:44:14.192972 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-854ca2bb-25d5-4c8e-8af3-c2a9c4fda256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:44:14.339710 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1115/2242] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:44:14 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:44:14.351792 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e45f521-ded7-427e-bb8e-6a44b9cf589d admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:44:14.354218 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-3e45f521-ded7-427e-bb8e-6a44b9cf589d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:44:14.487354 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e45f521-ded7-427e-bb8e-6a44b9cf589d admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:44:14.491891 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe53c736-ad69-48a6-a81f-1d64c1e35ec5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ba74e28ced4cbba857f54222dfbe41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:44:14.492520 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe53c736-ad69-48a6-a81f-1d64c1e35ec5 service cinder] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:14.492804 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe53c736-ad69-48a6-a81f-1d64c1e35ec5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:14.493221 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe53c736-ad69-48a6-a81f-1d64c1e35ec5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:14.599465 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe53c736-ad69-48a6-a81f-1d64c1e35ec5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:44:14.601268 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe53c736-ad69-48a6-a81f-1d64c1e35ec5 service cinder] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:44:14.613649 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1128/2243] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:44:14 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:44:15.042598 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd784c9-fba4-4d61-968f-ba18cb2f46e6 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:15.043434 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd784c9-fba4-4d61-968f-ba18cb2f46e6 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:15.043891 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd784c9-fba4-4d61-968f-ba18cb2f46e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:15.073522 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-1bd784c9-fba4-4d61-968f-ba18cb2f46e6 None None] MFA Rules not processed for user `1d346511f25c4f059f0dedfb3467ed06`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:44:15.075006 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-1bd784c9-fba4-4d61-968f-ba18cb2f46e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:44:15.085584 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:44:15.171638 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1116/2244] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:44:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:44:17.924536 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe53c736-ad69-48a6-a81f-1d64c1e35ec5 service cinder] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:44:17.935603 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe53c736-ad69-48a6-a81f-1d64c1e35ec5 service cinder] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:44:17.947306 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2007f009-4e61-4a4f-b96f-041a46e13a43 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ba74e28ced4cbba857f54222dfbe41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:44:17.947306 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2007f009-4e61-4a4f-b96f-041a46e13a43 service cinder] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:17.947306 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2007f009-4e61-4a4f-b96f-041a46e13a43 service cinder] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:17.947306 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2007f009-4e61-4a4f-b96f-041a46e13a43 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:18.021725 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:44:18.050736 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2007f009-4e61-4a4f-b96f-041a46e13a43 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:44:18.053279 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2007f009-4e61-4a4f-b96f-041a46e13a43 service cinder] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:44:18.064837 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1129/2245] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:44:17 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:44:18.752250 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61e5290-0b58-48b1-89fd-41d8608b24fd None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:18.752730 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61e5290-0b58-48b1-89fd-41d8608b24fd None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:18.752990 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61e5290-0b58-48b1-89fd-41d8608b24fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:18.784492 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-e61e5290-0b58-48b1-89fd-41d8608b24fd None None] MFA Rules not processed for user `4ec7ba61e65447a5a87b0f8fcc2b946c`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:44:18.785935 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-e61e5290-0b58-48b1-89fd-41d8608b24fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:44:18.893689 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1117/2246] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:44:18 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:44:19.135007 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716990d0-5385-4ed5-9d3f-d6bdce946986 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:19.135511 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716990d0-5385-4ed5-9d3f-d6bdce946986 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:19.135797 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716990d0-5385-4ed5-9d3f-d6bdce946986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:19.176219 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-716990d0-5385-4ed5-9d3f-d6bdce946986 None None] MFA Rules not processed for user `956ccf8932b24a7f8d74f4cbc3ead23a`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:44:19.178781 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-716990d0-5385-4ed5-9d3f-d6bdce946986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:44:19.262583 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1130/2247] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:44:19 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:44:19.267567 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61e5290-0b58-48b1-89fd-41d8608b24fd None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:44:19.347104 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61e5290-0b58-48b1-89fd-41d8608b24fd None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:44:19.351763 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fc8d0e-4eea-4d92-9be1-88fa7f5fff3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:44:19.352403 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fc8d0e-4eea-4d92-9be1-88fa7f5fff3f admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:19.352687 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fc8d0e-4eea-4d92-9be1-88fa7f5fff3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:19.353017 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fc8d0e-4eea-4d92-9be1-88fa7f5fff3f admin admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:19.353798 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fc8d0e-4eea-4d92-9be1-88fa7f5fff3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:44:19.356001 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fc8d0e-4eea-4d92-9be1-88fa7f5fff3f admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:44:19.474076 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1118/2248] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 08:44:19 2022] POST /identity/v3/projects => generated 377 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:44:19.475249 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716990d0-5385-4ed5-9d3f-d6bdce946986 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:44:19.492435 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716990d0-5385-4ed5-9d3f-d6bdce946986 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:44:19.557522 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737d95c4-1b1c-4989-bd5c-05285d9413fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:44:19.558113 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737d95c4-1b1c-4989-bd5c-05285d9413fe admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:19.558390 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737d95c4-1b1c-4989-bd5c-05285d9413fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:19.558730 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737d95c4-1b1c-4989-bd5c-05285d9413fe admin admin] PATH_INFO: `/v3/users` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:19.559339 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737d95c4-1b1c-4989-bd5c-05285d9413fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:44:19.562039 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737d95c4-1b1c-4989-bd5c-05285d9413fe admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:44:19.646372 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1131/2249] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 08:44:19 2022] POST /identity/v3/users => generated 295 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:44:19.650511 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fc8d0e-4eea-4d92-9be1-88fa7f5fff3f admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:44:19.674122 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fc8d0e-4eea-4d92-9be1-88fa7f5fff3f admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:44:19.680478 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ec41ae-e1ea-4502-b264-4b61b59e1aba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:44:19.681436 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ec41ae-e1ea-4502-b264-4b61b59e1aba admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:19.681702 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ec41ae-e1ea-4502-b264-4b61b59e1aba admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:19.681973 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ec41ae-e1ea-4502-b264-4b61b59e1aba admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:19.682824 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ec41ae-e1ea-4502-b264-4b61b59e1aba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:44:19.684458 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ec41ae-e1ea-4502-b264-4b61b59e1aba admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:44:19.703300 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1119/2250] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:44:19 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:44:19.703793 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737d95c4-1b1c-4989-bd5c-05285d9413fe admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:44:19.715329 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737d95c4-1b1c-4989-bd5c-05285d9413fe admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:44:19.727162 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2adef0-a571-4914-9c30-21f3e8e28573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:44:19.727884 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2adef0-a571-4914-9c30-21f3e8e28573 admin admin] REQUEST_METHOD: `PUT` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:19.728168 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2adef0-a571-4914-9c30-21f3e8e28573 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:19.728436 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2adef0-a571-4914-9c30-21f3e8e28573 admin admin] PATH_INFO: `/v3/projects/80a657459f2b4b5db9f747c1e055ea77/users/144656bc6b3b43d783632e10927dd5c4/roles/ec3751692d9c4ecf81b4a71c936c46be` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:19.765007 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2adef0-a571-4914-9c30-21f3e8e28573 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80a657459f2b4b5db9f747c1e055ea77, user_id=144656bc6b3b43d783632e10927dd5c4, role_id=ec3751692d9c4ecf81b4a71c936c46be)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:44:19.768243 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2adef0-a571-4914-9c30-21f3e8e28573 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:44:19.880254 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1132/2251] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 08:44:19 2022] PUT /identity/v3/projects/80a657459f2b4b5db9f747c1e055ea77/users/144656bc6b3b43d783632e10927dd5c4/roles/ec3751692d9c4ecf81b4a71c936c46be => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:44:19.884360 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ec41ae-e1ea-4502-b264-4b61b59e1aba admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:44:19.903606 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ec41ae-e1ea-4502-b264-4b61b59e1aba admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:44:19.905822 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2adef0-a571-4914-9c30-21f3e8e28573 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:44:19.944562 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:44:19.960714 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:44:19.990118 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2adef0-a571-4914-9c30-21f3e8e28573 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:44:19.996605 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1cc962-70cc-4ef5-ad65-587daaee08a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ba74e28ced4cbba857f54222dfbe41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:44:19.999287 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1cc962-70cc-4ef5-ad65-587daaee08a2 service cinder] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:19.999287 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1cc962-70cc-4ef5-ad65-587daaee08a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:19.999942 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1cc962-70cc-4ef5-ad65-587daaee08a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:20.007063 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cac874-1244-4dd9-8705-cd4ead615160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:44:20.009367 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cac874-1244-4dd9-8705-cd4ead615160 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:20.009662 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cac874-1244-4dd9-8705-cd4ead615160 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:20.009937 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cac874-1244-4dd9-8705-cd4ead615160 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:20.010694 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cac874-1244-4dd9-8705-cd4ead615160 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:44:20.012312 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cac874-1244-4dd9-8705-cd4ead615160 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:44:20.027302 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1120/2252] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:44:19 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 08:44:20.028564 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cac874-1244-4dd9-8705-cd4ead615160 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:44:20.043343 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cac874-1244-4dd9-8705-cd4ead615160 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:44:20.057971 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d4e0eb-853f-4a6a-83e2-82fb8b490908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:44:20.058657 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d4e0eb-853f-4a6a-83e2-82fb8b490908 admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:20.059001 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d4e0eb-853f-4a6a-83e2-82fb8b490908 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:20.059389 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d4e0eb-853f-4a6a-83e2-82fb8b490908 admin admin] PATH_INFO: `/v3/projects/80a657459f2b4b5db9f747c1e055ea77/users/144656bc6b3b43d783632e10927dd5c4/roles/ec3751692d9c4ecf81b4a71c936c46be` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:20.061988 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d4e0eb-853f-4a6a-83e2-82fb8b490908 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80a657459f2b4b5db9f747c1e055ea77, user_id=144656bc6b3b43d783632e10927dd5c4, role_id=ec3751692d9c4ecf81b4a71c936c46be)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:44:20.064829 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d4e0eb-853f-4a6a-83e2-82fb8b490908 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:44:20.088881 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1121/2253] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 08:44:20 2022] PUT /identity/v3/projects/80a657459f2b4b5db9f747c1e055ea77/users/144656bc6b3b43d783632e10927dd5c4/roles/ec3751692d9c4ecf81b4a71c936c46be => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:44:20.100639 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1cc962-70cc-4ef5-ad65-587daaee08a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:44:20.102221 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1cc962-70cc-4ef5-ad65-587daaee08a2 service cinder] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:44:20.115331 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1133/2254] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:44:19 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 08:44:26.169833 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d4e0eb-853f-4a6a-83e2-82fb8b490908 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:44:26.245708 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d4e0eb-853f-4a6a-83e2-82fb8b490908 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:44:26.250760 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d0ac23-d71f-4493-a13d-79b20cb8acfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:44:26.251393 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d0ac23-d71f-4493-a13d-79b20cb8acfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:26.251851 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d0ac23-d71f-4493-a13d-79b20cb8acfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:26.252314 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d0ac23-d71f-4493-a13d-79b20cb8acfd admin admin] PATH_INFO: `/v3/users/a1a172cfb48f42b295d9ff34f3d72598` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:26.253899 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d0ac23-d71f-4493-a13d-79b20cb8acfd admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1a172cfb48f42b295d9ff34f3d72598)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:44:26.255829 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d0ac23-d71f-4493-a13d-79b20cb8acfd admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:44:26.447257 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-93d0ac23-d71f-4493-a13d-79b20cb8acfd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1a172cfb48f42b295d9ff34f3d72598'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:44:26.462832 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-93d0ac23-d71f-4493-a13d-79b20cb8acfd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1a172cfb48f42b295d9ff34f3d72598'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:44:26.477325 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-93d0ac23-d71f-4493-a13d-79b20cb8acfd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1a172cfb48f42b295d9ff34f3d72598'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:44:26.478039 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-93d0ac23-d71f-4493-a13d-79b20cb8acfd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1a172cfb48f42b295d9ff34f3d72598'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:44:26.478696 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-93d0ac23-d71f-4493-a13d-79b20cb8acfd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1a172cfb48f42b295d9ff34f3d72598'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:44:26.540029 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1122/2255] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 08:44:26 2022] DELETE /identity/v3/users/a1a172cfb48f42b295d9ff34f3d72598 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:44:26.549597 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bae564-ae6e-4171-8843-f4fe25ace016 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:26.549597 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bae564-ae6e-4171-8843-f4fe25ace016 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:26.549597 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bae564-ae6e-4171-8843-f4fe25ace016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:26.550516 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29407239-d75c-4d7f-bd5c-0664f16a36e7 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:26.550516 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29407239-d75c-4d7f-bd5c-0664f16a36e7 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:26.550516 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29407239-d75c-4d7f-bd5c-0664f16a36e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:26.574623 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-19bae564-ae6e-4171-8843-f4fe25ace016 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:44:26.574623 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-19bae564-ae6e-4171-8843-f4fe25ace016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:44:26.602281 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-29407239-d75c-4d7f-bd5c-0664f16a36e7 None None] MFA Rules not processed for user `144656bc6b3b43d783632e10927dd5c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:44:26.603841 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-29407239-d75c-4d7f-bd5c-0664f16a36e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:44:26.637388 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:44:26.664204 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1134/2256] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:44:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:44:26.671812 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bae564-ae6e-4171-8843-f4fe25ace016 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:44:26.673273 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-19bae564-ae6e-4171-8843-f4fe25ace016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:44:26.692964 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1123/2257] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:44:26 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:44:26.707315 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29407239-d75c-4d7f-bd5c-0664f16a36e7 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:44:26.708750 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-29407239-d75c-4d7f-bd5c-0664f16a36e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:44:26.744374 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bae564-ae6e-4171-8843-f4fe25ace016 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:44:26.748593 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed077e3-706d-4f02-a577-d9637173b775 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:44:26.749429 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed077e3-706d-4f02-a577-d9637173b775 service neutron] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:26.749863 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed077e3-706d-4f02-a577-d9637173b775 service neutron] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:26.750135 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed077e3-706d-4f02-a577-d9637173b775 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:26.760524 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed077e3-706d-4f02-a577-d9637173b775 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:44:26.761445 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed077e3-706d-4f02-a577-d9637173b775 service neutron] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:44:26.796632 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1135/2258] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:44:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 08:44:26.824574 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29407239-d75c-4d7f-bd5c-0664f16a36e7 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:44:26.833837 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83748fcd-89fd-4079-9ef2-26d90a5f5cab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a943008193914f76a663d489bd73e873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:44:26.834766 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83748fcd-89fd-4079-9ef2-26d90a5f5cab service nova] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:26.835100 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83748fcd-89fd-4079-9ef2-26d90a5f5cab service nova] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:26.835551 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83748fcd-89fd-4079-9ef2-26d90a5f5cab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:26.839498 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-83748fcd-89fd-4079-9ef2-26d90a5f5cab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:44:26.925313 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83748fcd-89fd-4079-9ef2-26d90a5f5cab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:44:26.926919 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83748fcd-89fd-4079-9ef2-26d90a5f5cab service nova] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:44:26.938067 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1124/2259] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:44:26 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 08:44:27.371880 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed077e3-706d-4f02-a577-d9637173b775 service neutron] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:44:27.375064 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-1ed077e3-706d-4f02-a577-d9637173b775 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:44:27.401295 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed077e3-706d-4f02-a577-d9637173b775 service neutron] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:44:27.412326 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783638e3-5941-4960-932d-a389709bf4e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:44:27.412874 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783638e3-5941-4960-932d-a389709bf4e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:27.413493 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783638e3-5941-4960-932d-a389709bf4e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:27.413803 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783638e3-5941-4960-932d-a389709bf4e5 admin admin] PATH_INFO: `/v3/projects/c44b7423b80d45c8b04f6df370115171` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:27.414909 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783638e3-5941-4960-932d-a389709bf4e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c44b7423b80d45c8b04f6df370115171)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:44:27.416884 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783638e3-5941-4960-932d-a389709bf4e5 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:44:27.442416 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:44:27.553995 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-783638e3-5941-4960-932d-a389709bf4e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c44b7423b80d45c8b04f6df370115171', 'request_id': 'req-783638e3-5941-4960-932d-a389709bf4e5'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:44:27.559355 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-783638e3-5941-4960-932d-a389709bf4e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c44b7423b80d45c8b04f6df370115171', 'request_id': 'req-783638e3-5941-4960-932d-a389709bf4e5'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:44:27.697767 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-783638e3-5941-4960-932d-a389709bf4e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c44b7423b80d45c8b04f6df370115171', 'request_id': 'req-783638e3-5941-4960-932d-a389709bf4e5'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:44:27.711130 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-783638e3-5941-4960-932d-a389709bf4e5 admin admin] The token cache is being invalidate because project c44b7423b80d45c8b04f6df370115171 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 08:44:27.711420 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-783638e3-5941-4960-932d-a389709bf4e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:44:27.712087 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-783638e3-5941-4960-932d-a389709bf4e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:44:27.713627 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1136/2260] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 08:44:27 2022] DELETE /identity/v3/projects/c44b7423b80d45c8b04f6df370115171 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:44:27.718027 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83748fcd-89fd-4079-9ef2-26d90a5f5cab service nova] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:44:27.729981 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83748fcd-89fd-4079-9ef2-26d90a5f5cab service nova] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:44:27.787809 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:44:27.792477 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49dcbe8-2651-459e-855b-f1388e65031c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:44:27.793151 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49dcbe8-2651-459e-855b-f1388e65031c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:27.793436 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49dcbe8-2651-459e-855b-f1388e65031c admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:27.794084 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49dcbe8-2651-459e-855b-f1388e65031c admin admin] PATH_INFO: `/v3/users/a7fdad6e74a64e44be962f62973b284d` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:27.794852 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49dcbe8-2651-459e-855b-f1388e65031c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7fdad6e74a64e44be962f62973b284d)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:44:27.797443 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49dcbe8-2651-459e-855b-f1388e65031c admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:44:27.956711 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-d49dcbe8-2651-459e-855b-f1388e65031c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7fdad6e74a64e44be962f62973b284d'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:44:27.969526 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-d49dcbe8-2651-459e-855b-f1388e65031c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7fdad6e74a64e44be962f62973b284d'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:44:27.976500 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-d49dcbe8-2651-459e-855b-f1388e65031c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7fdad6e74a64e44be962f62973b284d'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:44:27.977218 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-d49dcbe8-2651-459e-855b-f1388e65031c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7fdad6e74a64e44be962f62973b284d'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:44:27.977810 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-d49dcbe8-2651-459e-855b-f1388e65031c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7fdad6e74a64e44be962f62973b284d'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:44:28.055812 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1125/2261] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 08:44:27 2022] DELETE /identity/v3/users/a7fdad6e74a64e44be962f62973b284d => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:44:28.572308 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783638e3-5941-4960-932d-a389709bf4e5 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:44:28.572308 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-783638e3-5941-4960-932d-a389709bf4e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:44:28.584646 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783638e3-5941-4960-932d-a389709bf4e5 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:44:28.642961 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e4cd14-3d05-4992-854a-e2c5e685b12f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:44:28.643622 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e4cd14-3d05-4992-854a-e2c5e685b12f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:28.643897 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e4cd14-3d05-4992-854a-e2c5e685b12f admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:28.644165 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e4cd14-3d05-4992-854a-e2c5e685b12f admin admin] PATH_INFO: `/v3/projects/04070adc4317469794b066ccb48004bf` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:28.645376 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e4cd14-3d05-4992-854a-e2c5e685b12f admin admin] RBAC: Authorizing `identity:delete_project(project_id=04070adc4317469794b066ccb48004bf)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:44:28.647288 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e4cd14-3d05-4992-854a-e2c5e685b12f admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:44:28.783958 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-44e4cd14-3d05-4992-854a-e2c5e685b12f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04070adc4317469794b066ccb48004bf', 'request_id': 'req-44e4cd14-3d05-4992-854a-e2c5e685b12f'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:44:28.793633 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-44e4cd14-3d05-4992-854a-e2c5e685b12f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04070adc4317469794b066ccb48004bf', 'request_id': 'req-44e4cd14-3d05-4992-854a-e2c5e685b12f'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:44:28.857778 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-44e4cd14-3d05-4992-854a-e2c5e685b12f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04070adc4317469794b066ccb48004bf', 'request_id': 'req-44e4cd14-3d05-4992-854a-e2c5e685b12f'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:44:28.869209 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-44e4cd14-3d05-4992-854a-e2c5e685b12f admin admin] The token cache is being invalidate because project 04070adc4317469794b066ccb48004bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 08:44:28.869496 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-44e4cd14-3d05-4992-854a-e2c5e685b12f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:44:28.870147 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-44e4cd14-3d05-4992-854a-e2c5e685b12f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:44:28.871666 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1137/2262] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 08:44:28 2022] DELETE /identity/v3/projects/04070adc4317469794b066ccb48004bf => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:44:28.880640 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49dcbe8-2651-459e-855b-f1388e65031c admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:44:28.880640 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-d49dcbe8-2651-459e-855b-f1388e65031c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:44:28.891976 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49dcbe8-2651-459e-855b-f1388e65031c admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:44:28.949616 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a90e8d-33ed-4556-a473-c6a1104c2cc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:44:28.950300 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a90e8d-33ed-4556-a473-c6a1104c2cc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:28.950587 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a90e8d-33ed-4556-a473-c6a1104c2cc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:28.950859 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a90e8d-33ed-4556-a473-c6a1104c2cc6 admin admin] PATH_INFO: `/v3/users/1833ddad656c4e4ba2dabbaa36eb6b25` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:28.952114 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a90e8d-33ed-4556-a473-c6a1104c2cc6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1833ddad656c4e4ba2dabbaa36eb6b25)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:44:28.954195 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a90e8d-33ed-4556-a473-c6a1104c2cc6 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:44:29.191103 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-76a90e8d-33ed-4556-a473-c6a1104c2cc6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1833ddad656c4e4ba2dabbaa36eb6b25'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:44:29.211933 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-76a90e8d-33ed-4556-a473-c6a1104c2cc6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1833ddad656c4e4ba2dabbaa36eb6b25'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:44:29.219339 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-76a90e8d-33ed-4556-a473-c6a1104c2cc6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1833ddad656c4e4ba2dabbaa36eb6b25'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:44:29.220124 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-76a90e8d-33ed-4556-a473-c6a1104c2cc6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1833ddad656c4e4ba2dabbaa36eb6b25'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:44:29.222033 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-76a90e8d-33ed-4556-a473-c6a1104c2cc6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1833ddad656c4e4ba2dabbaa36eb6b25'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:44:29.226240 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:44:29.304068 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1126/2263] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 08:44:28 2022] DELETE /identity/v3/users/1833ddad656c4e4ba2dabbaa36eb6b25 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 08:44:29.751217 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e4cd14-3d05-4992-854a-e2c5e685b12f admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:44:29.754026 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-44e4cd14-3d05-4992-854a-e2c5e685b12f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:44:29.776101 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e4cd14-3d05-4992-854a-e2c5e685b12f admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:44:29.828147 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:44:29.858300 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317f7864-6886-4eb9-b186-5a8ca6f9aded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:44:29.859554 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317f7864-6886-4eb9-b186-5a8ca6f9aded admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:29.860058 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317f7864-6886-4eb9-b186-5a8ca6f9aded admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:29.860511 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317f7864-6886-4eb9-b186-5a8ca6f9aded admin admin] PATH_INFO: `/v3/projects/6bb6beb4c596446bb28b7941fe8fd785` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:29.862085 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317f7864-6886-4eb9-b186-5a8ca6f9aded admin admin] RBAC: Authorizing `identity:delete_project(project_id=6bb6beb4c596446bb28b7941fe8fd785)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:44:29.864724 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317f7864-6886-4eb9-b186-5a8ca6f9aded admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:44:29.985313 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-317f7864-6886-4eb9-b186-5a8ca6f9aded admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6bb6beb4c596446bb28b7941fe8fd785', 'request_id': 'req-317f7864-6886-4eb9-b186-5a8ca6f9aded'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:44:29.996567 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-317f7864-6886-4eb9-b186-5a8ca6f9aded admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6bb6beb4c596446bb28b7941fe8fd785', 'request_id': 'req-317f7864-6886-4eb9-b186-5a8ca6f9aded'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:44:30.086556 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-317f7864-6886-4eb9-b186-5a8ca6f9aded admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6bb6beb4c596446bb28b7941fe8fd785', 'request_id': 'req-317f7864-6886-4eb9-b186-5a8ca6f9aded'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:44:30.100224 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-317f7864-6886-4eb9-b186-5a8ca6f9aded admin admin] The token cache is being invalidate because project 6bb6beb4c596446bb28b7941fe8fd785 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 08:44:30.100825 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-317f7864-6886-4eb9-b186-5a8ca6f9aded admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:44:30.101661 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-317f7864-6886-4eb9-b186-5a8ca6f9aded admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:44:30.103565 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1138/2264] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 08:44:29 2022] DELETE /identity/v3/projects/6bb6beb4c596446bb28b7941fe8fd785 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:44:30.108669 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e972fc-d721-4412-b9bf-84e99a810124 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:30.109525 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e972fc-d721-4412-b9bf-84e99a810124 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:30.109804 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e972fc-d721-4412-b9bf-84e99a810124 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:30.131337 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-84e972fc-d721-4412-b9bf-84e99a810124 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:44:30.132886 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-84e972fc-d721-4412-b9bf-84e99a810124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:44:30.205407 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1127/2265] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:44:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:44:30.219928 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749a7236-8d8f-44b1-bf00-75fea75b5b2e None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:30.219928 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749a7236-8d8f-44b1-bf00-75fea75b5b2e None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:30.219928 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749a7236-8d8f-44b1-bf00-75fea75b5b2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:30.237694 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-749a7236-8d8f-44b1-bf00-75fea75b5b2e None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:44:30.238885 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-749a7236-8d8f-44b1-bf00-75fea75b5b2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:44:30.247853 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1139/2266] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:44:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:44:30.253474 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e972fc-d721-4412-b9bf-84e99a810124 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:44:30.266648 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e972fc-d721-4412-b9bf-84e99a810124 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:44:30.272853 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f43b60-ae4d-4fb7-8692-8ff02c4eed92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:44:30.272853 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f43b60-ae4d-4fb7-8692-8ff02c4eed92 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:30.272853 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f43b60-ae4d-4fb7-8692-8ff02c4eed92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:30.272853 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f43b60-ae4d-4fb7-8692-8ff02c4eed92 admin admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:30.273586 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f43b60-ae4d-4fb7-8692-8ff02c4eed92 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:44:30.273902 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f43b60-ae4d-4fb7-8692-8ff02c4eed92 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:44:30.275514 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f43b60-ae4d-4fb7-8692-8ff02c4eed92 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:44:30.288768 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1128/2267] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Aug 10 08:44:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:44:30.293567 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749a7236-8d8f-44b1-bf00-75fea75b5b2e None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:44:30.304469 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749a7236-8d8f-44b1-bf00-75fea75b5b2e None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:44:30.309449 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc5a4f4-f323-41b1-a23b-48746582ced5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:44:30.310057 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc5a4f4-f323-41b1-a23b-48746582ced5 admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:30.310337 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc5a4f4-f323-41b1-a23b-48746582ced5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:30.310608 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc5a4f4-f323-41b1-a23b-48746582ced5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:30.311236 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc5a4f4-f323-41b1-a23b-48746582ced5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:44:30.313184 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc5a4f4-f323-41b1-a23b-48746582ced5 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:44:30.420706 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1140/2268] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 08:44:30 2022] POST /identity/v3/projects => generated 373 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:44:30.422131 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f43b60-ae4d-4fb7-8692-8ff02c4eed92 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:44:30.447218 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f43b60-ae4d-4fb7-8692-8ff02c4eed92 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:44:30.551136 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72297907-e1a7-4b39-ab83-f7ce46e3624d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:44:30.553295 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72297907-e1a7-4b39-ab83-f7ce46e3624d admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:30.553586 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72297907-e1a7-4b39-ab83-f7ce46e3624d admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:30.553896 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72297907-e1a7-4b39-ab83-f7ce46e3624d admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:30.554591 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72297907-e1a7-4b39-ab83-f7ce46e3624d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:44:30.556568 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72297907-e1a7-4b39-ab83-f7ce46e3624d admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:44:30.650787 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1129/2269] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 08:44:30 2022] POST /identity/v3/users => generated 287 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:44:30.652055 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc5a4f4-f323-41b1-a23b-48746582ced5 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:44:30.663707 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc5a4f4-f323-41b1-a23b-48746582ced5 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:44:30.668568 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2c6d9b-5ee6-427a-8302-c58a216d75d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:44:30.669223 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2c6d9b-5ee6-427a-8302-c58a216d75d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:30.670096 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2c6d9b-5ee6-427a-8302-c58a216d75d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:30.670096 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2c6d9b-5ee6-427a-8302-c58a216d75d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:30.675575 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2c6d9b-5ee6-427a-8302-c58a216d75d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:44:30.677294 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2c6d9b-5ee6-427a-8302-c58a216d75d4 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:44:30.696949 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1141/2270] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:44:30 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 08:44:30.706596 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72297907-e1a7-4b39-ab83-f7ce46e3624d admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:44:30.717775 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72297907-e1a7-4b39-ab83-f7ce46e3624d admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:44:30.725261 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37f5780-e496-4512-8e76-c46313e9d0d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:44:30.725677 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37f5780-e496-4512-8e76-c46313e9d0d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:30.725840 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37f5780-e496-4512-8e76-c46313e9d0d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:30.726117 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37f5780-e496-4512-8e76-c46313e9d0d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:30.726777 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37f5780-e496-4512-8e76-c46313e9d0d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:44:30.728438 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37f5780-e496-4512-8e76-c46313e9d0d9 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:44:30.744266 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1130/2271] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:44:30 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 08:44:30.748951 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2c6d9b-5ee6-427a-8302-c58a216d75d4 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:44:30.765518 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2c6d9b-5ee6-427a-8302-c58a216d75d4 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:44:30.774560 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8ba5fe-b046-4a08-bbce-89a1fa33b684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:44:30.776097 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8ba5fe-b046-4a08-bbce-89a1fa33b684 admin admin] REQUEST_METHOD: `PUT` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:30.776688 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8ba5fe-b046-4a08-bbce-89a1fa33b684 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:30.777175 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8ba5fe-b046-4a08-bbce-89a1fa33b684 admin admin] PATH_INFO: `/v3/projects/ac310911aefb4e9280c2640b6a7b7914/users/08cd9e43b9b349e48c3cfab73e8a6bad/roles/44b64db484f24a46bf0b7ced46b519bf` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:30.799276 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8ba5fe-b046-4a08-bbce-89a1fa33b684 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac310911aefb4e9280c2640b6a7b7914, user_id=08cd9e43b9b349e48c3cfab73e8a6bad, role_id=44b64db484f24a46bf0b7ced46b519bf)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:44:30.802235 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8ba5fe-b046-4a08-bbce-89a1fa33b684 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:44:30.870800 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1142/2272] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 08:44:30 2022] PUT /identity/v3/projects/ac310911aefb4e9280c2640b6a7b7914/users/08cd9e43b9b349e48c3cfab73e8a6bad/roles/44b64db484f24a46bf0b7ced46b519bf => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:44:30.887937 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48433f62-0b41-4015-a751-e2509d26cee2 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:30.888220 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48433f62-0b41-4015-a751-e2509d26cee2 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:30.888961 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48433f62-0b41-4015-a751-e2509d26cee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:30.926666 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-48433f62-0b41-4015-a751-e2509d26cee2 None None] MFA Rules not processed for user `08cd9e43b9b349e48c3cfab73e8a6bad`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:44:30.928263 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-48433f62-0b41-4015-a751-e2509d26cee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:44:30.994510 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:44:31.019143 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1131/2273] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:44:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 08:44:31.027302 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8ba5fe-b046-4a08-bbce-89a1fa33b684 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:44:31.029082 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-3a8ba5fe-b046-4a08-bbce-89a1fa33b684 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:44:31.097533 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8ba5fe-b046-4a08-bbce-89a1fa33b684 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:44:31.101713 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08526bee-5aec-44ee-bdd6-eab22063810d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ba74e28ced4cbba857f54222dfbe41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:44:31.102329 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08526bee-5aec-44ee-bdd6-eab22063810d service cinder] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:31.102616 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08526bee-5aec-44ee-bdd6-eab22063810d service cinder] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:31.102878 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08526bee-5aec-44ee-bdd6-eab22063810d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:31.112865 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08526bee-5aec-44ee-bdd6-eab22063810d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:44:31.114653 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08526bee-5aec-44ee-bdd6-eab22063810d service cinder] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:44:31.125255 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1143/2274] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:44:31 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:44:31.155724 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48433f62-0b41-4015-a751-e2509d26cee2 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:44:31.233274 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48433f62-0b41-4015-a751-e2509d26cee2 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:44:31.240817 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92bd3c1-2ed3-4b64-9eb2-5f04df3fd2dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:44:31.241603 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92bd3c1-2ed3-4b64-9eb2-5f04df3fd2dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:31.242857 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92bd3c1-2ed3-4b64-9eb2-5f04df3fd2dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:31.243241 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92bd3c1-2ed3-4b64-9eb2-5f04df3fd2dc admin admin] PATH_INFO: `/v3/users/08cd9e43b9b349e48c3cfab73e8a6bad` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:31.244514 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92bd3c1-2ed3-4b64-9eb2-5f04df3fd2dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=08cd9e43b9b349e48c3cfab73e8a6bad)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:44:31.246698 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92bd3c1-2ed3-4b64-9eb2-5f04df3fd2dc admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:44:31.427725 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-a92bd3c1-2ed3-4b64-9eb2-5f04df3fd2dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08cd9e43b9b349e48c3cfab73e8a6bad'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:44:31.440823 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-a92bd3c1-2ed3-4b64-9eb2-5f04df3fd2dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08cd9e43b9b349e48c3cfab73e8a6bad'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:44:31.448078 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-a92bd3c1-2ed3-4b64-9eb2-5f04df3fd2dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08cd9e43b9b349e48c3cfab73e8a6bad'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:44:31.448896 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-a92bd3c1-2ed3-4b64-9eb2-5f04df3fd2dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08cd9e43b9b349e48c3cfab73e8a6bad'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:44:31.449543 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-a92bd3c1-2ed3-4b64-9eb2-5f04df3fd2dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08cd9e43b9b349e48c3cfab73e8a6bad'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:44:31.522227 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1132/2275] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 08:44:31 2022] DELETE /identity/v3/users/08cd9e43b9b349e48c3cfab73e8a6bad => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:44:31.527219 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3349cd92-af08-4e27-806a-9cd879787fb9 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:31.527537 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3349cd92-af08-4e27-806a-9cd879787fb9 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:31.527818 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3349cd92-af08-4e27-806a-9cd879787fb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:31.547213 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:44:31.554295 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-3349cd92-af08-4e27-806a-9cd879787fb9 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:44:31.555750 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-3349cd92-af08-4e27-806a-9cd879787fb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:44:31.618432 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1144/2276] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:44:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:44:31.629775 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92bd3c1-2ed3-4b64-9eb2-5f04df3fd2dc admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:44:31.631393 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-a92bd3c1-2ed3-4b64-9eb2-5f04df3fd2dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:44:31.724511 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92bd3c1-2ed3-4b64-9eb2-5f04df3fd2dc admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:44:31.728058 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12bbd97-375f-4e5c-a9af-1c59dff532ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:44:31.728666 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12bbd97-375f-4e5c-a9af-1c59dff532ab service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:31.730083 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12bbd97-375f-4e5c-a9af-1c59dff532ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:31.730363 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12bbd97-375f-4e5c-a9af-1c59dff532ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:31.741948 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12bbd97-375f-4e5c-a9af-1c59dff532ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:44:31.743498 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12bbd97-375f-4e5c-a9af-1c59dff532ab service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:44:31.760871 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1133/2277] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:44:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:44:32.317382 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3349cd92-af08-4e27-806a-9cd879787fb9 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:44:32.318940 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-3349cd92-af08-4e27-806a-9cd879787fb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:44:32.342593 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3349cd92-af08-4e27-806a-9cd879787fb9 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:44:32.349635 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397ce89c-2324-4702-a7e7-ba98ca37f826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:44:32.350293 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397ce89c-2324-4702-a7e7-ba98ca37f826 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:32.350621 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397ce89c-2324-4702-a7e7-ba98ca37f826 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:32.350864 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397ce89c-2324-4702-a7e7-ba98ca37f826 admin admin] PATH_INFO: `/v3/projects/ac310911aefb4e9280c2640b6a7b7914` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:32.351994 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397ce89c-2324-4702-a7e7-ba98ca37f826 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac310911aefb4e9280c2640b6a7b7914)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:44:32.353984 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397ce89c-2324-4702-a7e7-ba98ca37f826 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:44:32.470958 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-397ce89c-2324-4702-a7e7-ba98ca37f826 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac310911aefb4e9280c2640b6a7b7914', 'request_id': 'req-397ce89c-2324-4702-a7e7-ba98ca37f826'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:44:32.479465 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-397ce89c-2324-4702-a7e7-ba98ca37f826 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac310911aefb4e9280c2640b6a7b7914', 'request_id': 'req-397ce89c-2324-4702-a7e7-ba98ca37f826'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:44:32.561221 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-397ce89c-2324-4702-a7e7-ba98ca37f826 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac310911aefb4e9280c2640b6a7b7914', 'request_id': 'req-397ce89c-2324-4702-a7e7-ba98ca37f826'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:44:32.576746 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-397ce89c-2324-4702-a7e7-ba98ca37f826 admin admin] The token cache is being invalidate because project ac310911aefb4e9280c2640b6a7b7914 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 08:44:32.577183 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-397ce89c-2324-4702-a7e7-ba98ca37f826 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:44:32.577937 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-397ce89c-2324-4702-a7e7-ba98ca37f826 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:44:32.582116 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1145/2278] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 08:44:32 2022] DELETE /identity/v3/projects/ac310911aefb4e9280c2640b6a7b7914 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:44:41.757298 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=83,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:44:42.570960 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=51,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:44:45.459607 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12bbd97-375f-4e5c-a9af-1c59dff532ab service neutron] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:44:45.462368 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-b12bbd97-375f-4e5c-a9af-1c59dff532ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:44:45.530956 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12bbd97-375f-4e5c-a9af-1c59dff532ab service neutron] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:44:45.591664 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116c5787-8a90-4e92-8e56-148b799c51e9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2912307c56594125b157f1cfdf7c96a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:44:45.592264 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116c5787-8a90-4e92-8e56-148b799c51e9 service placement] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:45.592542 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116c5787-8a90-4e92-8e56-148b799c51e9 service placement] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:45.592812 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116c5787-8a90-4e92-8e56-148b799c51e9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:45.594537 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-116c5787-8a90-4e92-8e56-148b799c51e9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:44:45.606471 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116c5787-8a90-4e92-8e56-148b799c51e9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:44:45.608030 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116c5787-8a90-4e92-8e56-148b799c51e9 service placement] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:44:45.618781 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1134/2279] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:44:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:44:55.614725 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=58 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:44:56.206296 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9453a1-e59d-4210-8785-1f62980c5a63 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:44:56.206837 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9453a1-e59d-4210-8785-1f62980c5a63 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:44:56.206933 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9453a1-e59d-4210-8785-1f62980c5a63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:44:56.225744 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-fb9453a1-e59d-4210-8785-1f62980c5a63 None None] MFA Rules not processed for user `956ccf8932b24a7f8d74f4cbc3ead23a`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:44:56.226937 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-fb9453a1-e59d-4210-8785-1f62980c5a63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:44:56.281768 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1146/2280] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:44:56 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:45:04.803475 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116c5787-8a90-4e92-8e56-148b799c51e9 service placement] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:45:04.804927 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-116c5787-8a90-4e92-8e56-148b799c51e9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:45:04.862455 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116c5787-8a90-4e92-8e56-148b799c51e9 service placement] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:45:04.866590 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea7fcfe-6e9c-4b39-b381-96be3f890621 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:45:04.867187 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea7fcfe-6e9c-4b39-b381-96be3f890621 service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:45:04.867461 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea7fcfe-6e9c-4b39-b381-96be3f890621 service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:45:04.867728 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea7fcfe-6e9c-4b39-b381-96be3f890621 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:45:04.869503 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-2ea7fcfe-6e9c-4b39-b381-96be3f890621 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:45:04.932570 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea7fcfe-6e9c-4b39-b381-96be3f890621 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:45:04.934088 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea7fcfe-6e9c-4b39-b381-96be3f890621 service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:45:04.944059 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1135/2281] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:45:04 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:45:06.274846 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=25 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:45:14.939298 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=50 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:45:20.500775 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9453a1-e59d-4210-8785-1f62980c5a63 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:45:20.511564 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9453a1-e59d-4210-8785-1f62980c5a63 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:45:20.515048 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b007f12-e83b-4b77-b9ec-d52da9129de2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:45:20.515637 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b007f12-e83b-4b77-b9ec-d52da9129de2 service neutron] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:45:20.515910 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b007f12-e83b-4b77-b9ec-d52da9129de2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:45:20.516177 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b007f12-e83b-4b77-b9ec-d52da9129de2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:45:20.517895 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-9b007f12-e83b-4b77-b9ec-d52da9129de2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:45:20.582419 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b007f12-e83b-4b77-b9ec-d52da9129de2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:45:20.584625 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b007f12-e83b-4b77-b9ec-d52da9129de2 service neutron] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:45:20.594604 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1147/2282] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:45:20 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:45:25.758870 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d753d68-c49a-4251-af36-eb5ae00f2552 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:45:25.759367 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d753d68-c49a-4251-af36-eb5ae00f2552 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:45:25.759484 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d753d68-c49a-4251-af36-eb5ae00f2552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:45:25.780471 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-9d753d68-c49a-4251-af36-eb5ae00f2552 None None] MFA Rules not processed for user `258ec0f975eb47738f4dac356c0578e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:45:25.781714 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-9d753d68-c49a-4251-af36-eb5ae00f2552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:45:25.835926 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1136/2283] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:45:25 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:45:30.590068 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=30 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:45:35.828655 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=25 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:45:40.740690 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09737524-a69f-4d03-b4c5-0930ec635d42 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:45:40.741370 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09737524-a69f-4d03-b4c5-0930ec635d42 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:45:40.741370 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09737524-a69f-4d03-b4c5-0930ec635d42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:45:40.760937 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-09737524-a69f-4d03-b4c5-0930ec635d42 None None] MFA Rules not processed for user `c140d31bd2b84c48a170e3ca90a041ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:45:40.762137 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-09737524-a69f-4d03-b4c5-0930ec635d42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:45:40.819279 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1148/2284] 10.222.0.57 () {58 vars in 1009 bytes} [Wed Aug 10 08:45:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:45:40.958244 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d753d68-c49a-4251-af36-eb5ae00f2552 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:45:40.960006 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-9d753d68-c49a-4251-af36-eb5ae00f2552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:45:41.029463 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d753d68-c49a-4251-af36-eb5ae00f2552 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:45:41.034349 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fdc082-f00c-4763-85a3-46e413d9f291 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4b26ba3a9e04c38b52db2a46198a53c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:45:41.034947 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fdc082-f00c-4763-85a3-46e413d9f291 service swift] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:45:41.035226 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fdc082-f00c-4763-85a3-46e413d9f291 service swift] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:45:41.035495 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fdc082-f00c-4763-85a3-46e413d9f291 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:45:41.043589 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fdc082-f00c-4763-85a3-46e413d9f291 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:45:41.045197 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fdc082-f00c-4763-85a3-46e413d9f291 service swift] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:45:41.054202 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1137/2285] 10.222.0.57 () {60 vars in 1384 bytes} [Wed Aug 10 08:45:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 08:45:50.811584 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=25 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:45:51.049882 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=32 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:45:59.374264 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af149696-4ddd-4de5-82e1-2d26eff68d5f None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:45:59.374792 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af149696-4ddd-4de5-82e1-2d26eff68d5f None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:45:59.374792 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af149696-4ddd-4de5-82e1-2d26eff68d5f None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:45:59.376142 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1149/2286] 10.222.0.57 () {54 vars in 921 bytes} [Wed Aug 10 08:45:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:45:59.382794 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617ad773-8cca-49bb-a847-1a3d9335b4e3 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:45:59.383299 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617ad773-8cca-49bb-a847-1a3d9335b4e3 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:45:59.383299 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617ad773-8cca-49bb-a847-1a3d9335b4e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:45:59.405653 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-617ad773-8cca-49bb-a847-1a3d9335b4e3 None None] MFA Rules not processed for user `a943008193914f76a663d489bd73e873`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:45:59.406876 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-617ad773-8cca-49bb-a847-1a3d9335b4e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:45:59.477850 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1138/2287] 10.222.0.57 () {58 vars in 1018 bytes} [Wed Aug 10 08:45:59 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:45:59.489505 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af149696-4ddd-4de5-82e1-2d26eff68d5f None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:45:59.492124 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-af149696-4ddd-4de5-82e1-2d26eff68d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:45:59.548390 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af149696-4ddd-4de5-82e1-2d26eff68d5f None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:45:59.551959 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150421d0-1458-43ab-b2b0-111a43b8def0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3263af99fc2f48e79fd1e5b8a1e555a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:45:59.552553 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150421d0-1458-43ab-b2b0-111a43b8def0 service glance] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:45:59.553389 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150421d0-1458-43ab-b2b0-111a43b8def0 service glance] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:45:59.553657 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150421d0-1458-43ab-b2b0-111a43b8def0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:45:59.563452 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150421d0-1458-43ab-b2b0-111a43b8def0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:45:59.564874 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150421d0-1458-43ab-b2b0-111a43b8def0 service glance] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:45:59.576219 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1150/2288] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:45:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:45:59.796190 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10229d99-9b92-4a83-ae67-10500a278e56 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:45:59.796492 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10229d99-9b92-4a83-ae67-10500a278e56 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:45:59.796763 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10229d99-9b92-4a83-ae67-10500a278e56 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:45:59.798157 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1139/2289] 10.222.0.57 () {54 vars in 921 bytes} [Wed Aug 10 08:45:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:45:59.802985 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8cc343-d36f-459e-bb00-c49d49c69e54 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:45:59.804500 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8cc343-d36f-459e-bb00-c49d49c69e54 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:45:59.804500 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8cc343-d36f-459e-bb00-c49d49c69e54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:45:59.823611 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-ba8cc343-d36f-459e-bb00-c49d49c69e54 None None] MFA Rules not processed for user `d4863ecff21e408c9deca645b88d3727`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:45:59.824822 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-ba8cc343-d36f-459e-bb00-c49d49c69e54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:45:59.831456 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1151/2290] 10.222.0.57 () {58 vars in 1018 bytes} [Wed Aug 10 08:45:59 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 08:45:59.843475 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10229d99-9b92-4a83-ae67-10500a278e56 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:45:59.845184 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-10229d99-9b92-4a83-ae67-10500a278e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:45:59.858799 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10229d99-9b92-4a83-ae67-10500a278e56 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:45:59.862388 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50534156-42b3-41df-9ba5-6fbf1481ac5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:45:59.862980 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50534156-42b3-41df-9ba5-6fbf1481ac5e service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:45:59.863259 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50534156-42b3-41df-9ba5-6fbf1481ac5e service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:45:59.863530 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50534156-42b3-41df-9ba5-6fbf1481ac5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:45:59.871248 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50534156-42b3-41df-9ba5-6fbf1481ac5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:45:59.872679 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50534156-42b3-41df-9ba5-6fbf1481ac5e service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:45:59.880744 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1140/2291] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:45:59 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:45:59.965975 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc938911-1fe5-4fc9-a8f2-e94be36bd170 None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:45:59.966268 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc938911-1fe5-4fc9-a8f2-e94be36bd170 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:45:59.966564 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc938911-1fe5-4fc9-a8f2-e94be36bd170 None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:45:59.967920 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1152/2292] 10.222.0.57 () {54 vars in 921 bytes} [Wed Aug 10 08:45:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:45:59.973217 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2594a4-d3f1-42c6-93e0-6ebae4a8ac2a None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:45:59.973511 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2594a4-d3f1-42c6-93e0-6ebae4a8ac2a None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:45:59.973804 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2594a4-d3f1-42c6-93e0-6ebae4a8ac2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:45:59.993962 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-cf2594a4-d3f1-42c6-93e0-6ebae4a8ac2a None None] MFA Rules not processed for user `a943008193914f76a663d489bd73e873`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:45:59.995147 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-cf2594a4-d3f1-42c6-93e0-6ebae4a8ac2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:46:00.004531 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1141/2293] 10.222.0.57 () {58 vars in 1018 bytes} [Wed Aug 10 08:45:59 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:46:00.014776 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc938911-1fe5-4fc9-a8f2-e94be36bd170 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:46:00.026019 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc938911-1fe5-4fc9-a8f2-e94be36bd170 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:46:00.029847 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc65bf30-3770-4baa-a522-77ea493937f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:46:00.030439 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc65bf30-3770-4baa-a522-77ea493937f1 service neutron] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:46:00.030715 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc65bf30-3770-4baa-a522-77ea493937f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:46:00.031010 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc65bf30-3770-4baa-a522-77ea493937f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:46:00.041826 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc65bf30-3770-4baa-a522-77ea493937f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:46:00.043299 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc65bf30-3770-4baa-a522-77ea493937f1 service neutron] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:46:00.053891 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1153/2294] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:46:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:46:04.159519 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e5d03c-93c9-4f27-a25a-31c5e8f59d34 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:46:04.160012 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e5d03c-93c9-4f27-a25a-31c5e8f59d34 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:46:04.160221 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e5d03c-93c9-4f27-a25a-31c5e8f59d34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:46:04.182191 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-03e5d03c-93c9-4f27-a25a-31c5e8f59d34 None None] MFA Rules not processed for user `c140d31bd2b84c48a170e3ca90a041ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:46:04.183409 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-03e5d03c-93c9-4f27-a25a-31c5e8f59d34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:46:04.190666 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1142/2295] 10.222.0.57 () {58 vars in 1009 bytes} [Wed Aug 10 08:46:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:46:04.202596 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc65bf30-3770-4baa-a522-77ea493937f1 service neutron] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:46:04.213814 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc65bf30-3770-4baa-a522-77ea493937f1 service neutron] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:46:04.218753 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca37539-2651-4489-92f1-2a87674b8f62 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4b26ba3a9e04c38b52db2a46198a53c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:46:04.220591 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca37539-2651-4489-92f1-2a87674b8f62 service swift] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:46:04.220591 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca37539-2651-4489-92f1-2a87674b8f62 service swift] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:46:04.220591 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca37539-2651-4489-92f1-2a87674b8f62 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:46:04.228283 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca37539-2651-4489-92f1-2a87674b8f62 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:46:04.229884 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca37539-2651-4489-92f1-2a87674b8f62 service swift] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:46:04.237886 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1154/2296] 10.222.0.57 () {60 vars in 1384 bytes} [Wed Aug 10 08:46:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 08:46:14.176208 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=43 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:46:14.234678 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=41 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:46:27.904137 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e5d03c-93c9-4f27-a25a-31c5e8f59d34 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:46:27.913411 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e5d03c-93c9-4f27-a25a-31c5e8f59d34 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:46:27.916997 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a27e8b7-a65a-4f40-9de0-04452d62e5a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:46:27.917586 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a27e8b7-a65a-4f40-9de0-04452d62e5a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:46:27.917864 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a27e8b7-a65a-4f40-9de0-04452d62e5a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:46:27.918131 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a27e8b7-a65a-4f40-9de0-04452d62e5a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:46:27.920164 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-1a27e8b7-a65a-4f40-9de0-04452d62e5a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:46:27.933537 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a27e8b7-a65a-4f40-9de0-04452d62e5a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:46:27.935447 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a27e8b7-a65a-4f40-9de0-04452d62e5a2 service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:46:27.946440 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1143/2297] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:46:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:46:31.420738 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca37539-2651-4489-92f1-2a87674b8f62 service swift] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:46:31.422159 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-fca37539-2651-4489-92f1-2a87674b8f62 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:46:31.480664 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca37539-2651-4489-92f1-2a87674b8f62 service swift] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:46:31.484411 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377a08ad-9755-4cc1-9b7a-296508c939f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ba74e28ced4cbba857f54222dfbe41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:46:31.485056 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377a08ad-9755-4cc1-9b7a-296508c939f3 service cinder] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:46:31.485457 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377a08ad-9755-4cc1-9b7a-296508c939f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:46:31.485734 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377a08ad-9755-4cc1-9b7a-296508c939f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:46:31.489519 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-377a08ad-9755-4cc1-9b7a-296508c939f3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:46:31.503700 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377a08ad-9755-4cc1-9b7a-296508c939f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:46:31.505349 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377a08ad-9755-4cc1-9b7a-296508c939f3 service cinder] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:46:31.517006 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1155/2298] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:46:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:46:36.315077 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e5da5d-cbbf-44ce-9e01-e86e121e115d None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:46:36.315578 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e5da5d-cbbf-44ce-9e01-e86e121e115d None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:46:36.315673 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e5da5d-cbbf-44ce-9e01-e86e121e115d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:46:36.334954 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-64e5da5d-cbbf-44ce-9e01-e86e121e115d None None] MFA Rules not processed for user `c140d31bd2b84c48a170e3ca90a041ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:46:36.336246 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-64e5da5d-cbbf-44ce-9e01-e86e121e115d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:46:36.344647 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1144/2299] 10.222.0.57 () {58 vars in 967 bytes} [Wed Aug 10 08:46:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:46:36.357102 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377a08ad-9755-4cc1-9b7a-296508c939f3 service cinder] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:46:36.368572 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377a08ad-9755-4cc1-9b7a-296508c939f3 service cinder] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:46:36.373484 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4a1ad1-14b9-47ed-8d99-8a33ff0cbce8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4b26ba3a9e04c38b52db2a46198a53c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:46:36.374123 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4a1ad1-14b9-47ed-8d99-8a33ff0cbce8 service swift] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:46:36.374403 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4a1ad1-14b9-47ed-8d99-8a33ff0cbce8 service swift] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:46:36.374677 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4a1ad1-14b9-47ed-8d99-8a33ff0cbce8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:46:36.383109 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4a1ad1-14b9-47ed-8d99-8a33ff0cbce8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:46:36.384741 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4a1ad1-14b9-47ed-8d99-8a33ff0cbce8 service swift] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:46:36.394111 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1156/2300] 10.222.0.57 () {60 vars in 1384 bytes} [Wed Aug 10 08:46:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 08:46:46.330169 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=9 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:46:46.390444 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=32 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:46:47.885438 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e5da5d-cbbf-44ce-9e01-e86e121e115d None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:46:47.886815 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-64e5da5d-cbbf-44ce-9e01-e86e121e115d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:46:47.898587 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e5da5d-cbbf-44ce-9e01-e86e121e115d None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:46:47.902116 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2055fdf1-147c-4854-bb88-5f8fddea0185 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3263af99fc2f48e79fd1e5b8a1e555a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:46:47.902702 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2055fdf1-147c-4854-bb88-5f8fddea0185 service glance] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:46:47.902978 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2055fdf1-147c-4854-bb88-5f8fddea0185 service glance] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:46:47.903247 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2055fdf1-147c-4854-bb88-5f8fddea0185 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:46:47.904893 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-2055fdf1-147c-4854-bb88-5f8fddea0185 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:46:47.962898 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2055fdf1-147c-4854-bb88-5f8fddea0185 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:46:47.964245 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2055fdf1-147c-4854-bb88-5f8fddea0185 service glance] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:46:47.973261 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1145/2301] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:46:47 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:46:53.053997 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3579b4b-7828-47bd-98c1-7f76d34c533b None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:46:53.054513 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3579b4b-7828-47bd-98c1-7f76d34c533b None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:46:53.054619 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3579b4b-7828-47bd-98c1-7f76d34c533b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:46:53.073389 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-e3579b4b-7828-47bd-98c1-7f76d34c533b None None] MFA Rules not processed for user `4ec7ba61e65447a5a87b0f8fcc2b946c`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:46:53.074581 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-e3579b4b-7828-47bd-98c1-7f76d34c533b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:46:53.082048 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1157/2302] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:46:53 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:46:57.969235 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=28 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:47:01.699512 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2055fdf1-147c-4854-bb88-5f8fddea0185 service glance] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:47:01.710242 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2055fdf1-147c-4854-bb88-5f8fddea0185 service glance] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:47:01.713788 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bd1d1e-a717-4fcb-96f5-0c6032fdac9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:47:01.714376 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bd1d1e-a717-4fcb-96f5-0c6032fdac9a service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:47:01.714647 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bd1d1e-a717-4fcb-96f5-0c6032fdac9a service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:47:01.714914 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bd1d1e-a717-4fcb-96f5-0c6032fdac9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:47:01.716588 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-60bd1d1e-a717-4fcb-96f5-0c6032fdac9a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:47:01.726332 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bd1d1e-a717-4fcb-96f5-0c6032fdac9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:47:01.727734 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bd1d1e-a717-4fcb-96f5-0c6032fdac9a service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:47:01.736483 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1146/2303] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:47:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:47:03.068601 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=3 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:47:03.295341 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3579b4b-7828-47bd-98c1-7f76d34c533b None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:47:03.296881 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-e3579b4b-7828-47bd-98c1-7f76d34c533b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:47:03.312393 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3579b4b-7828-47bd-98c1-7f76d34c533b None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:47:03.317828 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157e160f-5d67-49c6-9da6-6ffe5ee94ccc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a943008193914f76a663d489bd73e873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:47:03.318413 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157e160f-5d67-49c6-9da6-6ffe5ee94ccc service nova] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:47:03.318744 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157e160f-5d67-49c6-9da6-6ffe5ee94ccc service nova] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:47:03.318961 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157e160f-5d67-49c6-9da6-6ffe5ee94ccc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:47:03.320630 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-157e160f-5d67-49c6-9da6-6ffe5ee94ccc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:47:03.377167 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157e160f-5d67-49c6-9da6-6ffe5ee94ccc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:47:03.378748 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157e160f-5d67-49c6-9da6-6ffe5ee94ccc service nova] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:47:03.388341 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1158/2304] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:47:03 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:47:11.732061 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=6 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:47:13.384383 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=28 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:47:16.325672 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bd1d1e-a717-4fcb-96f5-0c6032fdac9a service neutron] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:47:16.327188 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-60bd1d1e-a717-4fcb-96f5-0c6032fdac9a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:47:16.343108 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bd1d1e-a717-4fcb-96f5-0c6032fdac9a service neutron] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:47:16.349504 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb3769d-f86a-4189-af4f-1790832c5649 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a943008193914f76a663d489bd73e873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:47:16.350096 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb3769d-f86a-4189-af4f-1790832c5649 service nova] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:47:16.350370 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb3769d-f86a-4189-af4f-1790832c5649 service nova] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:47:16.350657 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb3769d-f86a-4189-af4f-1790832c5649 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:47:16.352338 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-6bb3769d-f86a-4189-af4f-1790832c5649 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:47:16.363098 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb3769d-f86a-4189-af4f-1790832c5649 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:47:16.364993 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb3769d-f86a-4189-af4f-1790832c5649 service nova] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:47:16.373754 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1147/2305] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:47:16 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:47:20.208729 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157e160f-5d67-49c6-9da6-6ffe5ee94ccc service nova] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:47:20.220219 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157e160f-5d67-49c6-9da6-6ffe5ee94ccc service nova] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:47:20.226038 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c5f2cc-d410-47a3-9093-8404659ad990 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a943008193914f76a663d489bd73e873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:47:20.226639 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c5f2cc-d410-47a3-9093-8404659ad990 service nova] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:47:20.226914 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c5f2cc-d410-47a3-9093-8404659ad990 service nova] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:47:20.227184 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c5f2cc-d410-47a3-9093-8404659ad990 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:47:20.228965 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-c7c5f2cc-d410-47a3-9093-8404659ad990 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:47:20.245241 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c5f2cc-d410-47a3-9093-8404659ad990 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:47:20.246898 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c5f2cc-d410-47a3-9093-8404659ad990 service nova] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:47:20.259052 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1159/2306] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:47:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 08:47:26.369668 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=6 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:47:30.253043 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=6 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:47:33.122750 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a3a78e-97df-4aa3-a2e1-d6bda2502a2e None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:47:33.123282 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a3a78e-97df-4aa3-a2e1-d6bda2502a2e None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:47:33.123379 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a3a78e-97df-4aa3-a2e1-d6bda2502a2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:47:33.143006 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-72a3a78e-97df-4aa3-a2e1-d6bda2502a2e None None] MFA Rules not processed for user `258ec0f975eb47738f4dac356c0578e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:47:33.144197 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-72a3a78e-97df-4aa3-a2e1-d6bda2502a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:47:33.151987 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1148/2307] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:47:33 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:47:33.455514 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c5f2cc-d410-47a3-9093-8404659ad990 service nova] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:47:33.470803 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c5f2cc-d410-47a3-9093-8404659ad990 service nova] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:47:33.470803 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ecf50a-3673-4d3c-98c4-ee5a064d2e94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:47:33.470803 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ecf50a-3673-4d3c-98c4-ee5a064d2e94 service neutron] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:47:33.470803 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ecf50a-3673-4d3c-98c4-ee5a064d2e94 service neutron] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:47:33.471227 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ecf50a-3673-4d3c-98c4-ee5a064d2e94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:47:33.472636 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-49ecf50a-3673-4d3c-98c4-ee5a064d2e94 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:47:33.482675 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ecf50a-3673-4d3c-98c4-ee5a064d2e94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:47:33.484189 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ecf50a-3673-4d3c-98c4-ee5a064d2e94 service neutron] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:47:33.493261 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1160/2308] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:47:33 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:47:33.916272 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a3a78e-97df-4aa3-a2e1-d6bda2502a2e None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:47:33.917657 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-72a3a78e-97df-4aa3-a2e1-d6bda2502a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:47:33.933565 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a3a78e-97df-4aa3-a2e1-d6bda2502a2e None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:47:33.938625 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a871751a-11a8-48b9-98be-023aeecab88f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2912307c56594125b157f1cfdf7c96a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:47:33.939313 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a871751a-11a8-48b9-98be-023aeecab88f service placement] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:47:33.939656 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a871751a-11a8-48b9-98be-023aeecab88f service placement] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:47:33.939990 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a871751a-11a8-48b9-98be-023aeecab88f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:47:33.941763 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-a871751a-11a8-48b9-98be-023aeecab88f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:47:33.963058 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a871751a-11a8-48b9-98be-023aeecab88f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:47:33.964645 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a871751a-11a8-48b9-98be-023aeecab88f service placement] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:47:33.975625 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1149/2309] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:47:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:47:43.488626 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=6 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:47:43.970699 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=9 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:47:45.594387 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ecf50a-3673-4d3c-98c4-ee5a064d2e94 service neutron] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:47:45.607847 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ecf50a-3673-4d3c-98c4-ee5a064d2e94 service neutron] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:47:45.613880 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b472a4-86b8-471d-9d02-66ecf6adad88 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a943008193914f76a663d489bd73e873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:47:45.614480 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b472a4-86b8-471d-9d02-66ecf6adad88 service nova] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:47:45.614759 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b472a4-86b8-471d-9d02-66ecf6adad88 service nova] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:47:45.615031 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b472a4-86b8-471d-9d02-66ecf6adad88 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:47:45.617316 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-e6b472a4-86b8-471d-9d02-66ecf6adad88 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:47:45.629235 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b472a4-86b8-471d-9d02-66ecf6adad88 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:47:45.630803 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b472a4-86b8-471d-9d02-66ecf6adad88 service nova] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:47:45.642033 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1161/2310] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:47:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:47:45.742572 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a871751a-11a8-48b9-98be-023aeecab88f service placement] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:47:45.754511 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a871751a-11a8-48b9-98be-023aeecab88f service placement] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:47:45.760743 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506cedf9-b843-4b2f-8733-16dc02c74d47 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a943008193914f76a663d489bd73e873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:47:45.761371 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506cedf9-b843-4b2f-8733-16dc02c74d47 service nova] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:47:45.761647 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506cedf9-b843-4b2f-8733-16dc02c74d47 service nova] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:47:45.761914 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506cedf9-b843-4b2f-8733-16dc02c74d47 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:47:45.763720 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-506cedf9-b843-4b2f-8733-16dc02c74d47 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:47:45.824621 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506cedf9-b843-4b2f-8733-16dc02c74d47 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:47:45.825604 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506cedf9-b843-4b2f-8733-16dc02c74d47 service nova] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:47:45.835311 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1150/2311] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:47:45 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:47:50.806561 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b472a4-86b8-471d-9d02-66ecf6adad88 service nova] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:47:50.818103 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b472a4-86b8-471d-9d02-66ecf6adad88 service nova] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:47:50.824907 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e40c39c-32aa-4079-8524-df5544064c4d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a943008193914f76a663d489bd73e873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:47:50.825753 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e40c39c-32aa-4079-8524-df5544064c4d service nova] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:47:50.826234 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e40c39c-32aa-4079-8524-df5544064c4d service nova] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:47:50.826505 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e40c39c-32aa-4079-8524-df5544064c4d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:47:50.828445 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-4e40c39c-32aa-4079-8524-df5544064c4d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:47:50.842575 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e40c39c-32aa-4079-8524-df5544064c4d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:47:50.844563 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e40c39c-32aa-4079-8524-df5544064c4d service nova] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:47:50.855800 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1162/2312] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:47:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:47:55.830636 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=28 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:47:57.670503 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2bc423-a102-4a09-9336-6ae9c1a05ff2 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:47:57.671119 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2bc423-a102-4a09-9336-6ae9c1a05ff2 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:47:57.671119 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2bc423-a102-4a09-9336-6ae9c1a05ff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:47:57.690967 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-aa2bc423-a102-4a09-9336-6ae9c1a05ff2 None None] MFA Rules not processed for user `3bf116fade674838b9219e15dd20eda2`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:47:57.692197 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-aa2bc423-a102-4a09-9336-6ae9c1a05ff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:47:57.700515 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1151/2313] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:47:57 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:47:58.057163 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e40c39c-32aa-4079-8524-df5544064c4d service nova] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:47:58.066376 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e40c39c-32aa-4079-8524-df5544064c4d service nova] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:47:58.070846 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46beae96-9126-4431-b614-9ba4fd7767db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:47:58.071535 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46beae96-9126-4431-b614-9ba4fd7767db service neutron] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:47:58.072026 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46beae96-9126-4431-b614-9ba4fd7767db service neutron] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:47:58.072422 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46beae96-9126-4431-b614-9ba4fd7767db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:47:58.074278 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-46beae96-9126-4431-b614-9ba4fd7767db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:47:58.085717 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46beae96-9126-4431-b614-9ba4fd7767db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:47:58.087230 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46beae96-9126-4431-b614-9ba4fd7767db service neutron] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:47:58.095182 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1163/2314] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:47:58 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:48:07.685486 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=3 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:48:08.091782 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=18 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:48:10.740881 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7bfa89-a3e6-4749-b0ce-4ec9f7c4dbc8 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:48:10.741364 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7bfa89-a3e6-4749-b0ce-4ec9f7c4dbc8 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:48:10.741473 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7bfa89-a3e6-4749-b0ce-4ec9f7c4dbc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:48:10.761155 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-aa7bfa89-a3e6-4749-b0ce-4ec9f7c4dbc8 None None] MFA Rules not processed for user `956ccf8932b24a7f8d74f4cbc3ead23a`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:48:10.762335 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-aa7bfa89-a3e6-4749-b0ce-4ec9f7c4dbc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:48:10.769088 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1152/2315] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:48:10 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:48:12.232730 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadbf5a4-88c3-4d38-86f2-6aa2475d0d4d None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:48:12.233695 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadbf5a4-88c3-4d38-86f2-6aa2475d0d4d None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:48:12.234166 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadbf5a4-88c3-4d38-86f2-6aa2475d0d4d None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:48:12.238876 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1164/2316] 10.222.0.57 () {54 vars in 921 bytes} [Wed Aug 10 08:48:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:48:12.243353 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b10246e-a232-44ae-a55a-7fab51f839ad None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:48:12.244190 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b10246e-a232-44ae-a55a-7fab51f839ad None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:48:12.244468 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b10246e-a232-44ae-a55a-7fab51f839ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:48:12.269701 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-9b10246e-a232-44ae-a55a-7fab51f839ad None None] MFA Rules not processed for user `d4863ecff21e408c9deca645b88d3727`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:48:12.276742 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-9b10246e-a232-44ae-a55a-7fab51f839ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:48:12.299185 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1153/2317] 10.222.0.57 () {58 vars in 1018 bytes} [Wed Aug 10 08:48:12 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:48:12.311520 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadbf5a4-88c3-4d38-86f2-6aa2475d0d4d None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:48:12.323654 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadbf5a4-88c3-4d38-86f2-6aa2475d0d4d None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:48:12.327584 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e338b0-d65b-439b-b57c-7aa73c2020bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:48:12.328578 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e338b0-d65b-439b-b57c-7aa73c2020bf service neutron] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:48:12.329079 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e338b0-d65b-439b-b57c-7aa73c2020bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:48:12.329663 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e338b0-d65b-439b-b57c-7aa73c2020bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:48:12.344794 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e338b0-d65b-439b-b57c-7aa73c2020bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:48:12.346362 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e338b0-d65b-439b-b57c-7aa73c2020bf service neutron] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:48:12.367028 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1165/2318] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:48:12 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:48:22.263183 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=6 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:48:22.351091 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=6 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:48:34.890220 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13be5548-42a2-4c4e-ae63-91b9433c1455 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:48:34.890731 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13be5548-42a2-4c4e-ae63-91b9433c1455 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:48:34.890842 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13be5548-42a2-4c4e-ae63-91b9433c1455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:48:34.909769 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-13be5548-42a2-4c4e-ae63-91b9433c1455 None None] MFA Rules not processed for user `9797df3e05944552b741f53773abf80a`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:48:34.912504 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-13be5548-42a2-4c4e-ae63-91b9433c1455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:48:34.918041 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1154/2319] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:48:34 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:48:36.787063 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e338b0-d65b-439b-b57c-7aa73c2020bf service neutron] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:48:36.797783 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e338b0-d65b-439b-b57c-7aa73c2020bf service neutron] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:48:36.806662 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0386bc48-cf5e-467c-9b10-388c5c67ca43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:48:36.807376 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0386bc48-cf5e-467c-9b10-388c5c67ca43 service neutron] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:48:36.807721 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0386bc48-cf5e-467c-9b10-388c5c67ca43 service neutron] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:48:36.808053 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0386bc48-cf5e-467c-9b10-388c5c67ca43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:48:36.810591 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-0386bc48-cf5e-467c-9b10-388c5c67ca43 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:48:36.886546 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0386bc48-cf5e-467c-9b10-388c5c67ca43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:48:36.888008 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0386bc48-cf5e-467c-9b10-388c5c67ca43 service neutron] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:48:36.900110 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1166/2320] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:48:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:48:40.759138 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dded86e9-1783-407c-901b-6e82486f347e None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:48:40.759877 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dded86e9-1783-407c-901b-6e82486f347e None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:48:40.759979 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dded86e9-1783-407c-901b-6e82486f347e None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:48:40.762869 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1155/2321] 10.222.0.57 () {54 vars in 921 bytes} [Wed Aug 10 08:48:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:48:40.802251 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd325d82-117a-4920-a536-823bfc2c6bd4 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:48:40.802834 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd325d82-117a-4920-a536-823bfc2c6bd4 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:48:40.802834 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd325d82-117a-4920-a536-823bfc2c6bd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:48:40.833050 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-bd325d82-117a-4920-a536-823bfc2c6bd4 None None] MFA Rules not processed for user `2912307c56594125b157f1cfdf7c96a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:48:40.834367 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-bd325d82-117a-4920-a536-823bfc2c6bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:48:40.844134 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1167/2322] 10.222.0.57 () {58 vars in 1018 bytes} [Wed Aug 10 08:48:40 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:48:40.856145 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dded86e9-1783-407c-901b-6e82486f347e None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:48:40.857833 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-dded86e9-1783-407c-901b-6e82486f347e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:48:40.880788 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dded86e9-1783-407c-901b-6e82486f347e None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:48:40.887310 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5ee6d6-ad20-4288-82ec-fbbc81bb55ba service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2912307c56594125b157f1cfdf7c96a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:48:40.887912 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5ee6d6-ad20-4288-82ec-fbbc81bb55ba service placement] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:48:40.888232 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5ee6d6-ad20-4288-82ec-fbbc81bb55ba service placement] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:48:40.888543 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5ee6d6-ad20-4288-82ec-fbbc81bb55ba service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:48:40.906459 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5ee6d6-ad20-4288-82ec-fbbc81bb55ba service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:48:40.911237 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5ee6d6-ad20-4288-82ec-fbbc81bb55ba service placement] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:48:40.912434 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd325d82-117a-4920-a536-823bfc2c6bd4 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:48:40.914777 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-bd325d82-117a-4920-a536-823bfc2c6bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:48:40.930890 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1156/2323] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:48:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:48:40.936904 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd325d82-117a-4920-a536-823bfc2c6bd4 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:48:40.942142 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea5c885-c453-4654-be54-4d2264160ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:48:40.942904 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea5c885-c453-4654-be54-4d2264160ee2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:48:40.943289 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea5c885-c453-4654-be54-4d2264160ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:48:40.943802 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea5c885-c453-4654-be54-4d2264160ee2 admin admin] PATH_INFO: `/v3/users/4ec7ba61e65447a5a87b0f8fcc2b946c` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:48:40.945109 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea5c885-c453-4654-be54-4d2264160ee2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ec7ba61e65447a5a87b0f8fcc2b946c)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:48:40.947479 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea5c885-c453-4654-be54-4d2264160ee2 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:48:41.120335 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-2ea5c885-c453-4654-be54-4d2264160ee2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ec7ba61e65447a5a87b0f8fcc2b946c'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:48:41.135864 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-2ea5c885-c453-4654-be54-4d2264160ee2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ec7ba61e65447a5a87b0f8fcc2b946c'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:48:41.143865 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-2ea5c885-c453-4654-be54-4d2264160ee2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ec7ba61e65447a5a87b0f8fcc2b946c'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:48:41.195672 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-2ea5c885-c453-4654-be54-4d2264160ee2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ec7ba61e65447a5a87b0f8fcc2b946c'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:48:41.196398 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-2ea5c885-c453-4654-be54-4d2264160ee2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ec7ba61e65447a5a87b0f8fcc2b946c'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:48:41.203598 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1168/2324] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 08:48:40 2022] DELETE /identity/v3/users/4ec7ba61e65447a5a87b0f8fcc2b946c => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:48:41.461485 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5ee6d6-ad20-4288-82ec-fbbc81bb55ba service placement] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:48:41.478099 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5ee6d6-ad20-4288-82ec-fbbc81bb55ba service placement] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:48:41.549006 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d63004-1a63-475c-9114-707a8709044f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:48:41.552178 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d63004-1a63-475c-9114-707a8709044f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:48:41.552178 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d63004-1a63-475c-9114-707a8709044f admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:48:41.552178 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d63004-1a63-475c-9114-707a8709044f admin admin] PATH_INFO: `/v3/projects/aa5abc2b996a4056b005a992b5d0a7e9` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:48:41.552178 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d63004-1a63-475c-9114-707a8709044f admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa5abc2b996a4056b005a992b5d0a7e9)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:48:41.553269 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d63004-1a63-475c-9114-707a8709044f admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:48:41.671674 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-e1d63004-1a63-475c-9114-707a8709044f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa5abc2b996a4056b005a992b5d0a7e9', 'request_id': 'req-e1d63004-1a63-475c-9114-707a8709044f'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:48:41.753750 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-e1d63004-1a63-475c-9114-707a8709044f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa5abc2b996a4056b005a992b5d0a7e9', 'request_id': 'req-e1d63004-1a63-475c-9114-707a8709044f'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:48:41.763657 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-e1d63004-1a63-475c-9114-707a8709044f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa5abc2b996a4056b005a992b5d0a7e9', 'request_id': 'req-e1d63004-1a63-475c-9114-707a8709044f'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:48:41.773230 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-e1d63004-1a63-475c-9114-707a8709044f admin admin] The token cache is being invalidate because project aa5abc2b996a4056b005a992b5d0a7e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 08:48:41.773524 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-e1d63004-1a63-475c-9114-707a8709044f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:48:41.774208 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-e1d63004-1a63-475c-9114-707a8709044f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:48:41.775676 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1157/2325] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 08:48:41 2022] DELETE /identity/v3/projects/aa5abc2b996a4056b005a992b5d0a7e9 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:48:41.780644 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a49c08c-1011-4982-878f-bc4a42bdd033 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:48:41.786017 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a49c08c-1011-4982-878f-bc4a42bdd033 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:48:41.786017 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a49c08c-1011-4982-878f-bc4a42bdd033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:48:41.804455 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-4a49c08c-1011-4982-878f-bc4a42bdd033 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:48:41.805774 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-4a49c08c-1011-4982-878f-bc4a42bdd033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:48:41.875968 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:48:41.884945 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1169/2326] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:48:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 08:48:41.893009 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e41370-467a-44a4-8d61-8e87ad2e480f None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:48:41.893301 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e41370-467a-44a4-8d61-8e87ad2e480f None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:48:41.893578 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e41370-467a-44a4-8d61-8e87ad2e480f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:48:41.913871 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-a4e41370-467a-44a4-8d61-8e87ad2e480f None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:48:41.915082 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-a4e41370-467a-44a4-8d61-8e87ad2e480f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:48:41.924092 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1158/2327] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:48:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 08:48:41.928498 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a49c08c-1011-4982-878f-bc4a42bdd033 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:48:41.938655 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a49c08c-1011-4982-878f-bc4a42bdd033 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:48:41.944143 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e183350-2f07-436e-8fc6-4ab202691e5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:48:41.944755 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e183350-2f07-436e-8fc6-4ab202691e5f admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:48:41.945201 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e183350-2f07-436e-8fc6-4ab202691e5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:48:41.945476 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e183350-2f07-436e-8fc6-4ab202691e5f admin admin] PATH_INFO: `/v3/domains` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:48:41.946126 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e183350-2f07-436e-8fc6-4ab202691e5f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:48:41.946433 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e183350-2f07-436e-8fc6-4ab202691e5f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:48:41.948118 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e183350-2f07-436e-8fc6-4ab202691e5f admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:48:41.959828 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1170/2328] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Aug 10 08:48:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:48:41.964220 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e41370-467a-44a4-8d61-8e87ad2e480f None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:48:41.974913 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e41370-467a-44a4-8d61-8e87ad2e480f None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:48:41.980030 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cef55d-62f2-4738-bf36-05c09d8471a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:48:41.980641 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cef55d-62f2-4738-bf36-05c09d8471a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:48:41.980962 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cef55d-62f2-4738-bf36-05c09d8471a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:48:41.981406 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cef55d-62f2-4738-bf36-05c09d8471a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:48:41.982117 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cef55d-62f2-4738-bf36-05c09d8471a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:48:41.984499 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cef55d-62f2-4738-bf36-05c09d8471a5 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:48:42.091792 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1159/2329] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 08:48:41 2022] POST /identity/v3/projects => generated 387 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:48:42.093683 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e183350-2f07-436e-8fc6-4ab202691e5f admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:48:42.108111 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e183350-2f07-436e-8fc6-4ab202691e5f admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:48:42.178483 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d34249-9c7a-4131-9cc6-7cc8d26cf394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:48:42.179141 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d34249-9c7a-4131-9cc6-7cc8d26cf394 admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:48:42.179438 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d34249-9c7a-4131-9cc6-7cc8d26cf394 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:48:42.179710 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d34249-9c7a-4131-9cc6-7cc8d26cf394 admin admin] PATH_INFO: `/v3/users` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:48:42.180341 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d34249-9c7a-4131-9cc6-7cc8d26cf394 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:48:42.182328 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d34249-9c7a-4131-9cc6-7cc8d26cf394 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:48:42.263748 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1171/2330] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 08:48:42 2022] POST /identity/v3/users => generated 294 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 08:48:42.267998 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cef55d-62f2-4738-bf36-05c09d8471a5 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:48:42.279479 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cef55d-62f2-4738-bf36-05c09d8471a5 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:48:42.284828 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d26cb31-b82c-4b87-bf3d-bc937bb76bae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:48:42.285554 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d26cb31-b82c-4b87-bf3d-bc937bb76bae admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:48:42.285904 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d26cb31-b82c-4b87-bf3d-bc937bb76bae admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:48:42.286269 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d26cb31-b82c-4b87-bf3d-bc937bb76bae admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:48:42.286989 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d26cb31-b82c-4b87-bf3d-bc937bb76bae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:48:42.288703 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d26cb31-b82c-4b87-bf3d-bc937bb76bae admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:48:42.303038 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1160/2331] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:48:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:48:42.304241 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d34249-9c7a-4131-9cc6-7cc8d26cf394 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:48:42.315365 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d34249-9c7a-4131-9cc6-7cc8d26cf394 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:48:42.320064 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdb2d86-8d60-44bd-a323-d8053183fb5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:48:42.320669 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdb2d86-8d60-44bd-a323-d8053183fb5e admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:48:42.320978 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdb2d86-8d60-44bd-a323-d8053183fb5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:48:42.321253 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdb2d86-8d60-44bd-a323-d8053183fb5e admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:48:42.321902 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdb2d86-8d60-44bd-a323-d8053183fb5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:48:42.323507 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdb2d86-8d60-44bd-a323-d8053183fb5e admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:48:42.337181 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1172/2332] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:48:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 08:48:42.341263 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d26cb31-b82c-4b87-bf3d-bc937bb76bae admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:48:42.353066 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d26cb31-b82c-4b87-bf3d-bc937bb76bae admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:48:42.358172 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d74eee-49ab-42b3-a742-84037cab9d27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:48:42.360053 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d74eee-49ab-42b3-a742-84037cab9d27 admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:48:42.360182 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d74eee-49ab-42b3-a742-84037cab9d27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:48:42.360551 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d74eee-49ab-42b3-a742-84037cab9d27 admin admin] PATH_INFO: `/v3/projects/e65fe739ea8141889bb2a2705b598778/users/aa0df4a3a99148f5b2b40afbcf6e1d58/roles/44b64db484f24a46bf0b7ced46b519bf` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:48:42.377096 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d74eee-49ab-42b3-a742-84037cab9d27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e65fe739ea8141889bb2a2705b598778, user_id=aa0df4a3a99148f5b2b40afbcf6e1d58, role_id=44b64db484f24a46bf0b7ced46b519bf)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:48:42.380012 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d74eee-49ab-42b3-a742-84037cab9d27 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:48:42.464197 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1161/2333] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 08:48:42 2022] PUT /identity/v3/projects/e65fe739ea8141889bb2a2705b598778/users/aa0df4a3a99148f5b2b40afbcf6e1d58/roles/44b64db484f24a46bf0b7ced46b519bf => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 08:48:42.474862 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589dc5ed-b1a2-4863-80b4-c450df69185d None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:48:42.475147 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589dc5ed-b1a2-4863-80b4-c450df69185d None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:48:42.475446 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589dc5ed-b1a2-4863-80b4-c450df69185d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:48:42.504539 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-589dc5ed-b1a2-4863-80b4-c450df69185d None None] MFA Rules not processed for user `aa0df4a3a99148f5b2b40afbcf6e1d58`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:48:42.506046 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-589dc5ed-b1a2-4863-80b4-c450df69185d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:48:42.575127 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1173/2334] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:48:42 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:48:42.579778 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d74eee-49ab-42b3-a742-84037cab9d27 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:48:42.588904 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:48:42.647115 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d74eee-49ab-42b3-a742-84037cab9d27 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:48:42.651922 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8cca54-c81c-4767-8546-06d7620b7cdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:48:42.652527 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8cca54-c81c-4767-8546-06d7620b7cdd admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:48:42.652818 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8cca54-c81c-4767-8546-06d7620b7cdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:48:42.653155 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8cca54-c81c-4767-8546-06d7620b7cdd admin admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:48:42.653809 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8cca54-c81c-4767-8546-06d7620b7cdd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:48:42.655926 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8cca54-c81c-4767-8546-06d7620b7cdd admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:48:42.734916 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1162/2335] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 08:48:42 2022] POST /identity/v3/projects => generated 389 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:48:42.741415 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589dc5ed-b1a2-4863-80b4-c450df69185d None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:48:42.750730 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589dc5ed-b1a2-4863-80b4-c450df69185d None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:48:42.802817 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:48:42.809618 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45720ff3-0038-42e6-a418-0ad9fe493707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:48:42.810260 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45720ff3-0038-42e6-a418-0ad9fe493707 admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:48:42.810569 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45720ff3-0038-42e6-a418-0ad9fe493707 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:48:42.810861 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45720ff3-0038-42e6-a418-0ad9fe493707 admin admin] PATH_INFO: `/v3/users` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:48:42.812097 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45720ff3-0038-42e6-a418-0ad9fe493707 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:48:42.814219 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45720ff3-0038-42e6-a418-0ad9fe493707 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:48:42.895674 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1174/2336] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 08:48:42 2022] POST /identity/v3/users => generated 301 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:48:42.897344 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8cca54-c81c-4767-8546-06d7620b7cdd admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:48:42.908337 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8cca54-c81c-4767-8546-06d7620b7cdd admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:48:42.913302 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e623c0-64a8-40a6-bb3a-e792cfb8ea8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:48:42.914000 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e623c0-64a8-40a6-bb3a-e792cfb8ea8b admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:48:42.914334 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e623c0-64a8-40a6-bb3a-e792cfb8ea8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:48:42.914674 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e623c0-64a8-40a6-bb3a-e792cfb8ea8b admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:48:42.915396 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e623c0-64a8-40a6-bb3a-e792cfb8ea8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:48:42.917125 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e623c0-64a8-40a6-bb3a-e792cfb8ea8b admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:48:42.928686 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1163/2337] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:48:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:48:42.932771 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45720ff3-0038-42e6-a418-0ad9fe493707 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:48:42.944230 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45720ff3-0038-42e6-a418-0ad9fe493707 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:48:42.949440 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3ca0ba-517f-481a-95e1-43193a68f1b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:48:42.950100 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3ca0ba-517f-481a-95e1-43193a68f1b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:48:42.950381 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3ca0ba-517f-481a-95e1-43193a68f1b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:48:42.950663 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3ca0ba-517f-481a-95e1-43193a68f1b7 admin admin] PATH_INFO: `/v3/projects/bc581bbfe34045bf96ddeb463e954174/users/93ecda48313749329e38a2e2f05cda6e/roles/ec3751692d9c4ecf81b4a71c936c46be` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:48:42.967546 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3ca0ba-517f-481a-95e1-43193a68f1b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc581bbfe34045bf96ddeb463e954174, user_id=93ecda48313749329e38a2e2f05cda6e, role_id=ec3751692d9c4ecf81b4a71c936c46be)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:48:42.970287 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3ca0ba-517f-481a-95e1-43193a68f1b7 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:48:43.041148 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1175/2338] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 08:48:42 2022] PUT /identity/v3/projects/bc581bbfe34045bf96ddeb463e954174/users/93ecda48313749329e38a2e2f05cda6e/roles/ec3751692d9c4ecf81b4a71c936c46be => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:48:43.050978 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e623c0-64a8-40a6-bb3a-e792cfb8ea8b admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:48:43.065473 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e623c0-64a8-40a6-bb3a-e792cfb8ea8b admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:48:43.122920 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc47012-3590-47cf-96b7-195c3cf095fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:48:43.123519 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc47012-3590-47cf-96b7-195c3cf095fc admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:48:43.123798 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc47012-3590-47cf-96b7-195c3cf095fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:48:43.124345 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc47012-3590-47cf-96b7-195c3cf095fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:48:43.125027 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc47012-3590-47cf-96b7-195c3cf095fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:48:43.126624 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc47012-3590-47cf-96b7-195c3cf095fc admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:48:43.140598 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1164/2339] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:48:43 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:48:43.141910 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3ca0ba-517f-481a-95e1-43193a68f1b7 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:48:43.152135 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3ca0ba-517f-481a-95e1-43193a68f1b7 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:48:43.158065 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60cf4e6-1c64-4484-96fe-bc7f074111ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:48:43.158786 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60cf4e6-1c64-4484-96fe-bc7f074111ed admin admin] REQUEST_METHOD: `PUT` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:48:43.159134 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60cf4e6-1c64-4484-96fe-bc7f074111ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:48:43.159475 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60cf4e6-1c64-4484-96fe-bc7f074111ed admin admin] PATH_INFO: `/v3/projects/bc581bbfe34045bf96ddeb463e954174/users/93ecda48313749329e38a2e2f05cda6e/roles/ec3751692d9c4ecf81b4a71c936c46be` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:48:43.161461 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60cf4e6-1c64-4484-96fe-bc7f074111ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc581bbfe34045bf96ddeb463e954174, user_id=93ecda48313749329e38a2e2f05cda6e, role_id=ec3751692d9c4ecf81b4a71c936c46be)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:48:43.164208 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60cf4e6-1c64-4484-96fe-bc7f074111ed admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:48:43.174529 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1176/2340] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 08:48:43 2022] PUT /identity/v3/projects/bc581bbfe34045bf96ddeb463e954174/users/93ecda48313749329e38a2e2f05cda6e/roles/ec3751692d9c4ecf81b4a71c936c46be => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:48:43.186148 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0a9ad9-6c1b-40eb-af69-4bee94700b9c None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:48:43.188207 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0a9ad9-6c1b-40eb-af69-4bee94700b9c None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:48:43.188515 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0a9ad9-6c1b-40eb-af69-4bee94700b9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:48:43.217324 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-bf0a9ad9-6c1b-40eb-af69-4bee94700b9c None None] MFA Rules not processed for user `93ecda48313749329e38a2e2f05cda6e`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:48:43.218746 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-bf0a9ad9-6c1b-40eb-af69-4bee94700b9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:48:43.293815 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1165/2341] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:48:43 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:48:43.305296 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60cf4e6-1c64-4484-96fe-bc7f074111ed admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:48:43.307084 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-f60cf4e6-1c64-4484-96fe-bc7f074111ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:48:43.368083 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60cf4e6-1c64-4484-96fe-bc7f074111ed admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:48:43.372409 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ac8849-40b9-404f-a4dc-a5065d515f12 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ba74e28ced4cbba857f54222dfbe41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:48:43.373061 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac8849-40b9-404f-a4dc-a5065d515f12 service cinder] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:48:43.373443 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac8849-40b9-404f-a4dc-a5065d515f12 service cinder] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:48:43.373599 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac8849-40b9-404f-a4dc-a5065d515f12 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:48:43.388154 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ac8849-40b9-404f-a4dc-a5065d515f12 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:48:43.388154 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ac8849-40b9-404f-a4dc-a5065d515f12 service cinder] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:48:43.397691 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1177/2342] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:48:43 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:48:43.540670 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0a9ad9-6c1b-40eb-af69-4bee94700b9c None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:48:43.544991 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:48:43.604644 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0a9ad9-6c1b-40eb-af69-4bee94700b9c None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:48:43.610729 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa56ac9-77ea-40f5-a919-6f5c065ce9ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:48:43.611361 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa56ac9-77ea-40f5-a919-6f5c065ce9ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:48:43.611634 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa56ac9-77ea-40f5-a919-6f5c065ce9ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:48:43.611902 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa56ac9-77ea-40f5-a919-6f5c065ce9ff admin admin] PATH_INFO: `/v3/users/aa0df4a3a99148f5b2b40afbcf6e1d58` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:48:43.613063 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa56ac9-77ea-40f5-a919-6f5c065ce9ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa0df4a3a99148f5b2b40afbcf6e1d58)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:48:43.615083 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa56ac9-77ea-40f5-a919-6f5c065ce9ff admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:48:43.808189 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-3aa56ac9-77ea-40f5-a919-6f5c065ce9ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa0df4a3a99148f5b2b40afbcf6e1d58'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:48:43.820699 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-3aa56ac9-77ea-40f5-a919-6f5c065ce9ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa0df4a3a99148f5b2b40afbcf6e1d58'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:48:43.827867 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-3aa56ac9-77ea-40f5-a919-6f5c065ce9ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa0df4a3a99148f5b2b40afbcf6e1d58'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:48:43.828573 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-3aa56ac9-77ea-40f5-a919-6f5c065ce9ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa0df4a3a99148f5b2b40afbcf6e1d58'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:48:43.829208 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-3aa56ac9-77ea-40f5-a919-6f5c065ce9ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa0df4a3a99148f5b2b40afbcf6e1d58'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:48:43.908973 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1166/2343] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 08:48:43 2022] DELETE /identity/v3/users/aa0df4a3a99148f5b2b40afbcf6e1d58 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:48:43.913612 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efb5e69-ad23-496c-9ffb-32effcc07e21 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:48:43.914021 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efb5e69-ad23-496c-9ffb-32effcc07e21 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:48:43.914147 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efb5e69-ad23-496c-9ffb-32effcc07e21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:48:43.932788 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-5efb5e69-ad23-496c-9ffb-32effcc07e21 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:48:43.934005 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-5efb5e69-ad23-496c-9ffb-32effcc07e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:48:43.991301 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1178/2344] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:48:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 08:48:44.001118 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa56ac9-77ea-40f5-a919-6f5c065ce9ff admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:48:44.002590 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-3aa56ac9-77ea-40f5-a919-6f5c065ce9ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:48:44.060823 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa56ac9-77ea-40f5-a919-6f5c065ce9ff admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:48:44.064569 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937ed32f-a205-472f-825b-1aa66fea1dbf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:48:44.065203 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937ed32f-a205-472f-825b-1aa66fea1dbf service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:48:44.065480 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937ed32f-a205-472f-825b-1aa66fea1dbf service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:48:44.065749 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937ed32f-a205-472f-825b-1aa66fea1dbf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:48:44.076504 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937ed32f-a205-472f-825b-1aa66fea1dbf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:48:44.078043 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937ed32f-a205-472f-825b-1aa66fea1dbf service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:48:44.087999 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1167/2345] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:48:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:48:44.555134 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efb5e69-ad23-496c-9ffb-32effcc07e21 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:48:44.556611 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-5efb5e69-ad23-496c-9ffb-32effcc07e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:48:44.569198 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efb5e69-ad23-496c-9ffb-32effcc07e21 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:48:44.574020 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc4bc72-36a6-4329-9f1c-c383261c0475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:48:44.574661 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc4bc72-36a6-4329-9f1c-c383261c0475 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:48:44.574938 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc4bc72-36a6-4329-9f1c-c383261c0475 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:48:44.575209 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc4bc72-36a6-4329-9f1c-c383261c0475 admin admin] PATH_INFO: `/v3/projects/e65fe739ea8141889bb2a2705b598778` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:48:44.576416 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc4bc72-36a6-4329-9f1c-c383261c0475 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e65fe739ea8141889bb2a2705b598778)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:48:44.578899 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc4bc72-36a6-4329-9f1c-c383261c0475 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:48:44.688664 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:48:44.691526 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-3bc4bc72-36a6-4329-9f1c-c383261c0475 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e65fe739ea8141889bb2a2705b598778', 'request_id': 'req-3bc4bc72-36a6-4329-9f1c-c383261c0475'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:48:44.697058 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-3bc4bc72-36a6-4329-9f1c-c383261c0475 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e65fe739ea8141889bb2a2705b598778', 'request_id': 'req-3bc4bc72-36a6-4329-9f1c-c383261c0475'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:48:44.765414 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-3bc4bc72-36a6-4329-9f1c-c383261c0475 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e65fe739ea8141889bb2a2705b598778', 'request_id': 'req-3bc4bc72-36a6-4329-9f1c-c383261c0475'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:48:44.776968 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-3bc4bc72-36a6-4329-9f1c-c383261c0475 admin admin] The token cache is being invalidate because project e65fe739ea8141889bb2a2705b598778 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 08:48:44.777263 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-3bc4bc72-36a6-4329-9f1c-c383261c0475 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:48:44.777890 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-3bc4bc72-36a6-4329-9f1c-c383261c0475 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:48:44.779405 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1179/2346] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 08:48:44 2022] DELETE /identity/v3/projects/e65fe739ea8141889bb2a2705b598778 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:48:44.783585 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937ed32f-a205-472f-825b-1aa66fea1dbf service neutron] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:48:44.793273 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937ed32f-a205-472f-825b-1aa66fea1dbf service neutron] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:48:44.814011 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:48:44.851093 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1579b58-c006-4713-84c4-d5b3321fc525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:48:44.851714 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1579b58-c006-4713-84c4-d5b3321fc525 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:48:44.851988 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1579b58-c006-4713-84c4-d5b3321fc525 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:48:44.852251 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1579b58-c006-4713-84c4-d5b3321fc525 admin admin] PATH_INFO: `/v3/users/93ecda48313749329e38a2e2f05cda6e` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:48:44.853378 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1579b58-c006-4713-84c4-d5b3321fc525 admin admin] RBAC: Authorizing `identity:delete_user(user_id=93ecda48313749329e38a2e2f05cda6e)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:48:44.855277 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1579b58-c006-4713-84c4-d5b3321fc525 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:48:45.070615 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-a1579b58-c006-4713-84c4-d5b3321fc525 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93ecda48313749329e38a2e2f05cda6e'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:48:45.083208 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-a1579b58-c006-4713-84c4-d5b3321fc525 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93ecda48313749329e38a2e2f05cda6e'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:48:45.090203 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-a1579b58-c006-4713-84c4-d5b3321fc525 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93ecda48313749329e38a2e2f05cda6e'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:48:45.090880 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-a1579b58-c006-4713-84c4-d5b3321fc525 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93ecda48313749329e38a2e2f05cda6e'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:48:45.091488 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-a1579b58-c006-4713-84c4-d5b3321fc525 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93ecda48313749329e38a2e2f05cda6e'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:48:45.160628 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1168/2347] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 08:48:44 2022] DELETE /identity/v3/users/93ecda48313749329e38a2e2f05cda6e => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:48:45.596292 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc4bc72-36a6-4329-9f1c-c383261c0475 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:48:45.597796 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-3bc4bc72-36a6-4329-9f1c-c383261c0475 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:48:45.611986 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc4bc72-36a6-4329-9f1c-c383261c0475 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:48:45.671369 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d49e84e-2668-40f5-a508-c9bc48cab0df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:48:45.672005 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d49e84e-2668-40f5-a508-c9bc48cab0df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:48:45.672288 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d49e84e-2668-40f5-a508-c9bc48cab0df admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:48:45.672560 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d49e84e-2668-40f5-a508-c9bc48cab0df admin admin] PATH_INFO: `/v3/projects/bc581bbfe34045bf96ddeb463e954174` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:48:45.673717 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d49e84e-2668-40f5-a508-c9bc48cab0df admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc581bbfe34045bf96ddeb463e954174)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:48:45.675632 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d49e84e-2668-40f5-a508-c9bc48cab0df admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:48:45.772468 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-4d49e84e-2668-40f5-a508-c9bc48cab0df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc581bbfe34045bf96ddeb463e954174', 'request_id': 'req-4d49e84e-2668-40f5-a508-c9bc48cab0df'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:48:45.777882 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-4d49e84e-2668-40f5-a508-c9bc48cab0df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc581bbfe34045bf96ddeb463e954174', 'request_id': 'req-4d49e84e-2668-40f5-a508-c9bc48cab0df'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:48:45.845821 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-4d49e84e-2668-40f5-a508-c9bc48cab0df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc581bbfe34045bf96ddeb463e954174', 'request_id': 'req-4d49e84e-2668-40f5-a508-c9bc48cab0df'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:48:45.857149 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-4d49e84e-2668-40f5-a508-c9bc48cab0df admin admin] The token cache is being invalidate because project bc581bbfe34045bf96ddeb463e954174 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 08:48:45.857376 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-4d49e84e-2668-40f5-a508-c9bc48cab0df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:48:45.857991 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-4d49e84e-2668-40f5-a508-c9bc48cab0df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:48:45.860164 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1180/2348] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 08:48:45 2022] DELETE /identity/v3/projects/bc581bbfe34045bf96ddeb463e954174 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:48:45.864969 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c503d1bb-32b2-4c57-90e0-ad02726ce03a None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:48:45.865261 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c503d1bb-32b2-4c57-90e0-ad02726ce03a None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:48:45.867491 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c503d1bb-32b2-4c57-90e0-ad02726ce03a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:48:45.886660 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-c503d1bb-32b2-4c57-90e0-ad02726ce03a None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:48:45.887849 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-c503d1bb-32b2-4c57-90e0-ad02726ce03a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:48:45.947976 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1169/2349] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:48:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:48:45.955842 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8215f060-25ae-4f25-b8f2-8d24c84dfaa5 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:48:45.956152 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8215f060-25ae-4f25-b8f2-8d24c84dfaa5 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:48:45.956425 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8215f060-25ae-4f25-b8f2-8d24c84dfaa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:48:45.975623 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-8215f060-25ae-4f25-b8f2-8d24c84dfaa5 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:48:45.976824 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-8215f060-25ae-4f25-b8f2-8d24c84dfaa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:48:45.985168 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1181/2350] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:48:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:48:45.989365 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c503d1bb-32b2-4c57-90e0-ad02726ce03a None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:48:45.998922 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c503d1bb-32b2-4c57-90e0-ad02726ce03a None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:48:46.003986 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547d1609-2e00-4ae1-893b-d5e09d9ed1cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:48:46.004569 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547d1609-2e00-4ae1-893b-d5e09d9ed1cd admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:48:46.004844 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547d1609-2e00-4ae1-893b-d5e09d9ed1cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:48:46.005150 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547d1609-2e00-4ae1-893b-d5e09d9ed1cd admin admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:48:46.005791 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547d1609-2e00-4ae1-893b-d5e09d9ed1cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:48:46.006093 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547d1609-2e00-4ae1-893b-d5e09d9ed1cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:48:46.007669 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547d1609-2e00-4ae1-893b-d5e09d9ed1cd admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:48:46.019228 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1170/2351] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Aug 10 08:48:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 08:48:46.023438 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8215f060-25ae-4f25-b8f2-8d24c84dfaa5 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:48:46.032599 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8215f060-25ae-4f25-b8f2-8d24c84dfaa5 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:48:46.037226 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e54df19-2513-429b-b620-3c78ae803960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:48:46.037818 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e54df19-2513-429b-b620-3c78ae803960 admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:48:46.038095 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e54df19-2513-429b-b620-3c78ae803960 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:48:46.038365 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e54df19-2513-429b-b620-3c78ae803960 admin admin] PATH_INFO: `/v3/projects` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:48:46.038973 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e54df19-2513-429b-b620-3c78ae803960 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:48:46.040848 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e54df19-2513-429b-b620-3c78ae803960 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:48:46.121978 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1182/2352] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 08:48:46 2022] POST /identity/v3/projects => generated 387 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:48:46.127320 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547d1609-2e00-4ae1-893b-d5e09d9ed1cd admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:48:46.137672 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547d1609-2e00-4ae1-893b-d5e09d9ed1cd admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:48:46.166713 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:48:46.194167 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626aa2b6-6aec-4ec8-b5f7-917f082c8e71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:48:46.194775 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626aa2b6-6aec-4ec8-b5f7-917f082c8e71 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:48:46.195050 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626aa2b6-6aec-4ec8-b5f7-917f082c8e71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:48:46.195316 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626aa2b6-6aec-4ec8-b5f7-917f082c8e71 admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:48:46.195963 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626aa2b6-6aec-4ec8-b5f7-917f082c8e71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:48:46.197903 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626aa2b6-6aec-4ec8-b5f7-917f082c8e71 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:48:46.263347 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1171/2353] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 08:48:46 2022] POST /identity/v3/users => generated 294 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:48:46.267435 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e54df19-2513-429b-b620-3c78ae803960 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:48:46.278017 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e54df19-2513-429b-b620-3c78ae803960 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:48:46.284089 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a322f80-2918-46d9-a268-c55450b649b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:48:46.284703 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a322f80-2918-46d9-a268-c55450b649b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:48:46.285013 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a322f80-2918-46d9-a268-c55450b649b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:48:46.285283 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a322f80-2918-46d9-a268-c55450b649b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:48:46.285937 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a322f80-2918-46d9-a268-c55450b649b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:48:46.288731 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a322f80-2918-46d9-a268-c55450b649b4 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:48:46.298985 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1183/2354] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:48:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:48:46.303273 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626aa2b6-6aec-4ec8-b5f7-917f082c8e71 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:48:46.313229 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626aa2b6-6aec-4ec8-b5f7-917f082c8e71 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:48:46.317892 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d37fbc7-a867-46bf-b48d-5b531142e524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:48:46.318481 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d37fbc7-a867-46bf-b48d-5b531142e524 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:48:46.318757 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d37fbc7-a867-46bf-b48d-5b531142e524 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:48:46.319023 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d37fbc7-a867-46bf-b48d-5b531142e524 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:48:46.319662 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d37fbc7-a867-46bf-b48d-5b531142e524 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:48:46.321283 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d37fbc7-a867-46bf-b48d-5b531142e524 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:48:46.331996 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1172/2355] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:48:46 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:48:46.339756 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a322f80-2918-46d9-a268-c55450b649b4 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:48:46.350744 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a322f80-2918-46d9-a268-c55450b649b4 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:48:46.355505 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fd27e0-90c5-42d5-b18c-e27a7383717a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:48:46.356159 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd27e0-90c5-42d5-b18c-e27a7383717a admin admin] REQUEST_METHOD: `PUT` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:48:46.356438 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd27e0-90c5-42d5-b18c-e27a7383717a admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:48:46.356711 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd27e0-90c5-42d5-b18c-e27a7383717a admin admin] PATH_INFO: `/v3/projects/17f58c5626c84a858f2f922f2a0e0f3a/users/e191f323cbf84692b5519fd6b96c350e/roles/44b64db484f24a46bf0b7ced46b519bf` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:48:46.372716 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fd27e0-90c5-42d5-b18c-e27a7383717a admin admin] RBAC: Authorizing `identity:create_grant(project_id=17f58c5626c84a858f2f922f2a0e0f3a, user_id=e191f323cbf84692b5519fd6b96c350e, role_id=44b64db484f24a46bf0b7ced46b519bf)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:48:46.375591 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fd27e0-90c5-42d5-b18c-e27a7383717a admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:48:46.442106 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:48:46.447266 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1184/2356] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 08:48:46 2022] PUT /identity/v3/projects/17f58c5626c84a858f2f922f2a0e0f3a/users/e191f323cbf84692b5519fd6b96c350e/roles/44b64db484f24a46bf0b7ced46b519bf => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:48:46.459615 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef6a9ea-f520-4338-8e7b-703d1e2d2823 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:48:46.460314 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef6a9ea-f520-4338-8e7b-703d1e2d2823 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:48:46.460593 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef6a9ea-f520-4338-8e7b-703d1e2d2823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:48:46.489142 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-7ef6a9ea-f520-4338-8e7b-703d1e2d2823 None None] MFA Rules not processed for user `e191f323cbf84692b5519fd6b96c350e`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:48:46.490465 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-7ef6a9ea-f520-4338-8e7b-703d1e2d2823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:48:46.569210 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1173/2357] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:48:46 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:48:46.573615 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fd27e0-90c5-42d5-b18c-e27a7383717a admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:48:46.649758 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fd27e0-90c5-42d5-b18c-e27a7383717a admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:48:46.654831 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cedbb5-4d63-43c7-9d42-b71f3a8a705b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:48:46.656574 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cedbb5-4d63-43c7-9d42-b71f3a8a705b admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:48:46.656862 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cedbb5-4d63-43c7-9d42-b71f3a8a705b admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:48:46.657167 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cedbb5-4d63-43c7-9d42-b71f3a8a705b admin admin] PATH_INFO: `/v3/projects` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:48:46.657808 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cedbb5-4d63-43c7-9d42-b71f3a8a705b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:48:46.659880 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cedbb5-4d63-43c7-9d42-b71f3a8a705b admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:48:46.756166 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1185/2358] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 08:48:46 2022] POST /identity/v3/projects => generated 385 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:48:46.760387 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef6a9ea-f520-4338-8e7b-703d1e2d2823 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:48:46.771964 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef6a9ea-f520-4338-8e7b-703d1e2d2823 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:48:46.829382 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deabafc4-0b8e-47e6-9427-c6fd4f80e06f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:48:46.829993 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deabafc4-0b8e-47e6-9427-c6fd4f80e06f admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:48:46.830275 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deabafc4-0b8e-47e6-9427-c6fd4f80e06f admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:48:46.830548 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deabafc4-0b8e-47e6-9427-c6fd4f80e06f admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:48:46.831211 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deabafc4-0b8e-47e6-9427-c6fd4f80e06f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:48:46.833215 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deabafc4-0b8e-47e6-9427-c6fd4f80e06f admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:48:46.955504 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1174/2359] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 08:48:46 2022] POST /identity/v3/users => generated 299 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:48:46.959509 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cedbb5-4d63-43c7-9d42-b71f3a8a705b admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:48:46.970436 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cedbb5-4d63-43c7-9d42-b71f3a8a705b admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:48:46.974905 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2053a003-2851-4698-908e-5d1c0733fcdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:48:46.975496 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2053a003-2851-4698-908e-5d1c0733fcdb admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:48:46.975772 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2053a003-2851-4698-908e-5d1c0733fcdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:48:46.976038 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2053a003-2851-4698-908e-5d1c0733fcdb admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:48:46.976673 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2053a003-2851-4698-908e-5d1c0733fcdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:48:46.978349 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2053a003-2851-4698-908e-5d1c0733fcdb admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:48:46.988877 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1186/2360] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:48:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:48:46.997113 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deabafc4-0b8e-47e6-9427-c6fd4f80e06f admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:48:47.007573 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deabafc4-0b8e-47e6-9427-c6fd4f80e06f admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:48:47.012671 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5235ef4-dabc-458f-a4db-0cef02fee7cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:48:47.013358 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5235ef4-dabc-458f-a4db-0cef02fee7cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:48:47.013647 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5235ef4-dabc-458f-a4db-0cef02fee7cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:48:47.013922 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5235ef4-dabc-458f-a4db-0cef02fee7cb admin admin] PATH_INFO: `/v3/projects/1d00b164daa54338bc89f9fa76b1407b/users/44a32af45d6b418c88574ca14e7b875e/roles/ec3751692d9c4ecf81b4a71c936c46be` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:48:47.029736 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5235ef4-dabc-458f-a4db-0cef02fee7cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d00b164daa54338bc89f9fa76b1407b, user_id=44a32af45d6b418c88574ca14e7b875e, role_id=ec3751692d9c4ecf81b4a71c936c46be)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:48:47.032510 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5235ef4-dabc-458f-a4db-0cef02fee7cb admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:48:47.103790 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1175/2361] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 08:48:46 2022] PUT /identity/v3/projects/1d00b164daa54338bc89f9fa76b1407b/users/44a32af45d6b418c88574ca14e7b875e/roles/ec3751692d9c4ecf81b4a71c936c46be => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:48:47.107630 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2053a003-2851-4698-908e-5d1c0733fcdb admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:48:47.119566 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2053a003-2851-4698-908e-5d1c0733fcdb admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:48:47.175652 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ac139a-3f29-47bd-8b09-f1fe6f78a790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:48:47.176259 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ac139a-3f29-47bd-8b09-f1fe6f78a790 admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:48:47.176539 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ac139a-3f29-47bd-8b09-f1fe6f78a790 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:48:47.176809 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ac139a-3f29-47bd-8b09-f1fe6f78a790 admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:48:47.177487 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ac139a-3f29-47bd-8b09-f1fe6f78a790 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:48:47.179081 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ac139a-3f29-47bd-8b09-f1fe6f78a790 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:48:47.189221 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1187/2362] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:48:47 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:48:47.193375 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5235ef4-dabc-458f-a4db-0cef02fee7cb admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:48:47.200091 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:48:47.204489 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5235ef4-dabc-458f-a4db-0cef02fee7cb admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:48:47.210692 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a9b47d-3678-4f66-a214-ba9e6e1334cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:48:47.210692 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a9b47d-3678-4f66-a214-ba9e6e1334cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:48:47.210692 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a9b47d-3678-4f66-a214-ba9e6e1334cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:48:47.210692 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a9b47d-3678-4f66-a214-ba9e6e1334cf admin admin] PATH_INFO: `/v3/projects/1d00b164daa54338bc89f9fa76b1407b/users/44a32af45d6b418c88574ca14e7b875e/roles/ec3751692d9c4ecf81b4a71c936c46be` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:48:47.212180 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a9b47d-3678-4f66-a214-ba9e6e1334cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d00b164daa54338bc89f9fa76b1407b, user_id=44a32af45d6b418c88574ca14e7b875e, role_id=ec3751692d9c4ecf81b4a71c936c46be)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:48:47.214838 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a9b47d-3678-4f66-a214-ba9e6e1334cf admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:48:47.224767 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1176/2363] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 08:48:47 2022] PUT /identity/v3/projects/1d00b164daa54338bc89f9fa76b1407b/users/44a32af45d6b418c88574ca14e7b875e/roles/ec3751692d9c4ecf81b4a71c936c46be => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:48:47.234804 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260aa0d8-1fe1-4c1c-a0fb-9aa6e46424b7 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:48:47.239458 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260aa0d8-1fe1-4c1c-a0fb-9aa6e46424b7 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:48:47.239458 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260aa0d8-1fe1-4c1c-a0fb-9aa6e46424b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:48:47.262733 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-260aa0d8-1fe1-4c1c-a0fb-9aa6e46424b7 None None] MFA Rules not processed for user `44a32af45d6b418c88574ca14e7b875e`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:48:47.263929 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-260aa0d8-1fe1-4c1c-a0fb-9aa6e46424b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:48:47.321678 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:48:47.330913 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1188/2364] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:48:47 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 08:48:47.341798 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a9b47d-3678-4f66-a214-ba9e6e1334cf admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:48:47.343743 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-a3a9b47d-3678-4f66-a214-ba9e6e1334cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:48:47.401931 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a9b47d-3678-4f66-a214-ba9e6e1334cf admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:48:47.405717 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3419e2ae-47f1-4156-9b47-1628dc059375 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ba74e28ced4cbba857f54222dfbe41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:48:47.406242 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3419e2ae-47f1-4156-9b47-1628dc059375 service cinder] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:48:47.406522 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3419e2ae-47f1-4156-9b47-1628dc059375 service cinder] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:48:47.406795 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3419e2ae-47f1-4156-9b47-1628dc059375 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:48:47.415819 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3419e2ae-47f1-4156-9b47-1628dc059375 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:48:47.417283 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3419e2ae-47f1-4156-9b47-1628dc059375 service cinder] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:48:47.427366 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1177/2365] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:48:47 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:48:48.587476 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260aa0d8-1fe1-4c1c-a0fb-9aa6e46424b7 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:48:48.653246 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260aa0d8-1fe1-4c1c-a0fb-9aa6e46424b7 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:48:48.659277 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8766b939-2588-4595-91d6-9bcd87b275c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:48:48.660124 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8766b939-2588-4595-91d6-9bcd87b275c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:48:48.660403 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8766b939-2588-4595-91d6-9bcd87b275c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:48:48.660669 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8766b939-2588-4595-91d6-9bcd87b275c6 admin admin] PATH_INFO: `/v3/users/e191f323cbf84692b5519fd6b96c350e` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:48:48.661821 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8766b939-2588-4595-91d6-9bcd87b275c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e191f323cbf84692b5519fd6b96c350e)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:48:48.663759 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8766b939-2588-4595-91d6-9bcd87b275c6 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:48:48.891059 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-8766b939-2588-4595-91d6-9bcd87b275c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e191f323cbf84692b5519fd6b96c350e'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:48:48.904029 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-8766b939-2588-4595-91d6-9bcd87b275c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e191f323cbf84692b5519fd6b96c350e'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:48:48.913182 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-8766b939-2588-4595-91d6-9bcd87b275c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e191f323cbf84692b5519fd6b96c350e'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:48:48.989190 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-8766b939-2588-4595-91d6-9bcd87b275c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e191f323cbf84692b5519fd6b96c350e'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:48:48.993088 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-8766b939-2588-4595-91d6-9bcd87b275c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e191f323cbf84692b5519fd6b96c350e'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:48:48.996250 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1189/2366] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 08:48:48 2022] DELETE /identity/v3/users/e191f323cbf84692b5519fd6b96c350e => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:48:49.001124 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2dda55-b65f-44e0-b2fd-e417619f642a None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:48:49.001682 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2dda55-b65f-44e0-b2fd-e417619f642a None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:48:49.001859 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2dda55-b65f-44e0-b2fd-e417619f642a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:48:49.032104 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-ff2dda55-b65f-44e0-b2fd-e417619f642a None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:48:49.034154 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-ff2dda55-b65f-44e0-b2fd-e417619f642a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:48:49.093058 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1178/2367] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:48:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:48:49.103567 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8766b939-2588-4595-91d6-9bcd87b275c6 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:48:49.105002 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-8766b939-2588-4595-91d6-9bcd87b275c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:48:49.164088 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8766b939-2588-4595-91d6-9bcd87b275c6 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:48:49.167620 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dadded-1d4f-4b06-8aa1-9ff9529a6f67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:48:49.168213 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dadded-1d4f-4b06-8aa1-9ff9529a6f67 service neutron] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:48:49.168488 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dadded-1d4f-4b06-8aa1-9ff9529a6f67 service neutron] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:48:49.168756 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dadded-1d4f-4b06-8aa1-9ff9529a6f67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:48:49.177676 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dadded-1d4f-4b06-8aa1-9ff9529a6f67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:48:49.179081 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dadded-1d4f-4b06-8aa1-9ff9529a6f67 service neutron] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:48:49.188179 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1190/2368] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:48:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:48:49.762695 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2dda55-b65f-44e0-b2fd-e417619f642a None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:48:49.765986 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-ff2dda55-b65f-44e0-b2fd-e417619f642a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:48:49.779701 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2dda55-b65f-44e0-b2fd-e417619f642a None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:48:49.785357 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e83d541-4481-431a-87bf-bbe20dec6a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:48:49.786014 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e83d541-4481-431a-87bf-bbe20dec6a3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:48:49.786297 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e83d541-4481-431a-87bf-bbe20dec6a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:48:49.786626 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e83d541-4481-431a-87bf-bbe20dec6a3f admin admin] PATH_INFO: `/v3/projects/17f58c5626c84a858f2f922f2a0e0f3a` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:48:49.787841 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e83d541-4481-431a-87bf-bbe20dec6a3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=17f58c5626c84a858f2f922f2a0e0f3a)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:48:49.789816 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e83d541-4481-431a-87bf-bbe20dec6a3f admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:48:49.874648 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-8e83d541-4481-431a-87bf-bbe20dec6a3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17f58c5626c84a858f2f922f2a0e0f3a', 'request_id': 'req-8e83d541-4481-431a-87bf-bbe20dec6a3f'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:48:49.925950 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-8e83d541-4481-431a-87bf-bbe20dec6a3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17f58c5626c84a858f2f922f2a0e0f3a', 'request_id': 'req-8e83d541-4481-431a-87bf-bbe20dec6a3f'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:48:49.931627 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-8e83d541-4481-431a-87bf-bbe20dec6a3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17f58c5626c84a858f2f922f2a0e0f3a', 'request_id': 'req-8e83d541-4481-431a-87bf-bbe20dec6a3f'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:48:49.943136 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-8e83d541-4481-431a-87bf-bbe20dec6a3f admin admin] The token cache is being invalidate because project 17f58c5626c84a858f2f922f2a0e0f3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 08:48:49.943423 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-8e83d541-4481-431a-87bf-bbe20dec6a3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:48:49.944061 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-8e83d541-4481-431a-87bf-bbe20dec6a3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:48:49.945602 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1179/2369] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 08:48:49 2022] DELETE /identity/v3/projects/17f58c5626c84a858f2f922f2a0e0f3a => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 08:48:49.949586 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dadded-1d4f-4b06-8aa1-9ff9529a6f67 service neutron] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:48:49.959759 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dadded-1d4f-4b06-8aa1-9ff9529a6f67 service neutron] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:48:49.972197 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:48:50.015821 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8fefed-8372-4298-8dd4-d8bbdabcf0e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:48:50.016379 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8fefed-8372-4298-8dd4-d8bbdabcf0e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:48:50.016659 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8fefed-8372-4298-8dd4-d8bbdabcf0e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:48:50.016958 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8fefed-8372-4298-8dd4-d8bbdabcf0e0 admin admin] PATH_INFO: `/v3/users/44a32af45d6b418c88574ca14e7b875e` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:48:50.018786 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8fefed-8372-4298-8dd4-d8bbdabcf0e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=44a32af45d6b418c88574ca14e7b875e)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:48:50.020721 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8fefed-8372-4298-8dd4-d8bbdabcf0e0 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:48:50.209521 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-3a8fefed-8372-4298-8dd4-d8bbdabcf0e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44a32af45d6b418c88574ca14e7b875e'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:48:50.222845 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-3a8fefed-8372-4298-8dd4-d8bbdabcf0e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44a32af45d6b418c88574ca14e7b875e'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:48:50.233665 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-3a8fefed-8372-4298-8dd4-d8bbdabcf0e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44a32af45d6b418c88574ca14e7b875e'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:48:50.308652 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-3a8fefed-8372-4298-8dd4-d8bbdabcf0e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44a32af45d6b418c88574ca14e7b875e'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:48:50.309213 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-3a8fefed-8372-4298-8dd4-d8bbdabcf0e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44a32af45d6b418c88574ca14e7b875e'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:48:50.316035 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1191/2370] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 08:48:49 2022] DELETE /identity/v3/users/44a32af45d6b418c88574ca14e7b875e => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:48:50.749616 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e83d541-4481-431a-87bf-bbe20dec6a3f admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:48:50.751055 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-8e83d541-4481-431a-87bf-bbe20dec6a3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:48:50.765053 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e83d541-4481-431a-87bf-bbe20dec6a3f admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:48:50.779571 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:48:50.826213 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9291bb-0727-41cb-8017-252f2d6f420b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:48:50.826770 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9291bb-0727-41cb-8017-252f2d6f420b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:48:50.827050 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9291bb-0727-41cb-8017-252f2d6f420b admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:48:50.827327 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9291bb-0727-41cb-8017-252f2d6f420b admin admin] PATH_INFO: `/v3/projects/1d00b164daa54338bc89f9fa76b1407b` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:48:50.828450 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9291bb-0727-41cb-8017-252f2d6f420b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d00b164daa54338bc89f9fa76b1407b)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:48:50.831136 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9291bb-0727-41cb-8017-252f2d6f420b admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:48:50.953332 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-8b9291bb-0727-41cb-8017-252f2d6f420b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d00b164daa54338bc89f9fa76b1407b', 'request_id': 'req-8b9291bb-0727-41cb-8017-252f2d6f420b'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:48:51.005921 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-8b9291bb-0727-41cb-8017-252f2d6f420b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d00b164daa54338bc89f9fa76b1407b', 'request_id': 'req-8b9291bb-0727-41cb-8017-252f2d6f420b'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:48:51.011969 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-8b9291bb-0727-41cb-8017-252f2d6f420b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d00b164daa54338bc89f9fa76b1407b', 'request_id': 'req-8b9291bb-0727-41cb-8017-252f2d6f420b'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:48:51.024001 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-8b9291bb-0727-41cb-8017-252f2d6f420b admin admin] The token cache is being invalidate because project 1d00b164daa54338bc89f9fa76b1407b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 08:48:51.024284 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-8b9291bb-0727-41cb-8017-252f2d6f420b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:48:51.025005 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-8b9291bb-0727-41cb-8017-252f2d6f420b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:48:51.026674 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1180/2371] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 08:48:50 2022] DELETE /identity/v3/projects/1d00b164daa54338bc89f9fa76b1407b => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:48:51.031540 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653cf394-10c2-4e62-8502-f47ed6036751 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:48:51.032034 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653cf394-10c2-4e62-8502-f47ed6036751 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:48:51.032133 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653cf394-10c2-4e62-8502-f47ed6036751 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:48:51.051404 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-653cf394-10c2-4e62-8502-f47ed6036751 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:48:51.052596 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-653cf394-10c2-4e62-8502-f47ed6036751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:48:51.110813 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1192/2372] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:48:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:48:51.118710 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420a456f-86e5-40a6-82b5-67e8ea57047f None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:48:51.118992 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420a456f-86e5-40a6-82b5-67e8ea57047f None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:48:51.119264 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420a456f-86e5-40a6-82b5-67e8ea57047f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:48:51.139182 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-420a456f-86e5-40a6-82b5-67e8ea57047f None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:48:51.152609 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-420a456f-86e5-40a6-82b5-67e8ea57047f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:48:51.152609 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1181/2373] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:48:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:48:51.155550 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653cf394-10c2-4e62-8502-f47ed6036751 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:48:51.165351 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653cf394-10c2-4e62-8502-f47ed6036751 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:48:51.169910 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c0efe1-31f8-49de-a86d-318b76bf3280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:48:51.170506 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c0efe1-31f8-49de-a86d-318b76bf3280 admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:48:51.170784 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c0efe1-31f8-49de-a86d-318b76bf3280 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:48:51.171054 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c0efe1-31f8-49de-a86d-318b76bf3280 admin admin] PATH_INFO: `/v3/domains` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:48:51.171694 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c0efe1-31f8-49de-a86d-318b76bf3280 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:48:51.171997 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c0efe1-31f8-49de-a86d-318b76bf3280 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:48:51.174016 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c0efe1-31f8-49de-a86d-318b76bf3280 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:48:51.185443 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1193/2374] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Aug 10 08:48:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:48:51.190880 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420a456f-86e5-40a6-82b5-67e8ea57047f None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:48:51.200216 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420a456f-86e5-40a6-82b5-67e8ea57047f None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:48:51.205544 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c6504a-6289-45dd-8946-054e8a4bb500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:48:51.210149 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c6504a-6289-45dd-8946-054e8a4bb500 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:48:51.210149 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c6504a-6289-45dd-8946-054e8a4bb500 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:48:51.210149 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c6504a-6289-45dd-8946-054e8a4bb500 admin admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:48:51.210149 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c6504a-6289-45dd-8946-054e8a4bb500 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:48:51.210149 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c6504a-6289-45dd-8946-054e8a4bb500 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:48:51.328090 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1182/2375] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 08:48:51 2022] POST /identity/v3/projects => generated 383 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 08:48:51.333347 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c0efe1-31f8-49de-a86d-318b76bf3280 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:48:51.343339 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c0efe1-31f8-49de-a86d-318b76bf3280 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:48:51.365031 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:48:51.405736 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad41cd1f-0a0c-4623-bab1-d172a83f19ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:48:51.406352 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad41cd1f-0a0c-4623-bab1-d172a83f19ff admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:48:51.406630 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad41cd1f-0a0c-4623-bab1-d172a83f19ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:48:51.406901 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad41cd1f-0a0c-4623-bab1-d172a83f19ff admin admin] PATH_INFO: `/v3/users` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:48:51.407534 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad41cd1f-0a0c-4623-bab1-d172a83f19ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:48:51.409752 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad41cd1f-0a0c-4623-bab1-d172a83f19ff admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:48:51.483373 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1194/2376] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 08:48:51 2022] POST /identity/v3/users => generated 292 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:48:51.484752 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c6504a-6289-45dd-8946-054e8a4bb500 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:48:51.496751 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c6504a-6289-45dd-8946-054e8a4bb500 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:48:51.501403 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0033b0-c1ef-42e2-81a6-9f38a91b6ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:48:51.503177 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0033b0-c1ef-42e2-81a6-9f38a91b6ce8 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:48:51.503451 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0033b0-c1ef-42e2-81a6-9f38a91b6ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:48:51.503716 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0033b0-c1ef-42e2-81a6-9f38a91b6ce8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:48:51.504429 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0033b0-c1ef-42e2-81a6-9f38a91b6ce8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:48:51.506041 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0033b0-c1ef-42e2-81a6-9f38a91b6ce8 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:48:51.517140 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1183/2377] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:48:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:48:51.521084 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad41cd1f-0a0c-4623-bab1-d172a83f19ff admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:48:51.531395 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad41cd1f-0a0c-4623-bab1-d172a83f19ff admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:48:51.538470 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8014ec7e-5d44-4a4f-9117-abcc229d97d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:48:51.541073 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8014ec7e-5d44-4a4f-9117-abcc229d97d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:48:51.541073 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8014ec7e-5d44-4a4f-9117-abcc229d97d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:48:51.541073 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8014ec7e-5d44-4a4f-9117-abcc229d97d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:48:51.541073 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8014ec7e-5d44-4a4f-9117-abcc229d97d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:48:51.543108 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8014ec7e-5d44-4a4f-9117-abcc229d97d9 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:48:51.561095 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1195/2378] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:48:51 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:48:51.562998 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0033b0-c1ef-42e2-81a6-9f38a91b6ce8 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:48:51.573848 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0033b0-c1ef-42e2-81a6-9f38a91b6ce8 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:48:51.579100 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e50ec2d-6689-4e93-ab6a-58a56f374b83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:48:51.579751 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e50ec2d-6689-4e93-ab6a-58a56f374b83 admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:48:51.580032 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e50ec2d-6689-4e93-ab6a-58a56f374b83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:48:51.580304 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e50ec2d-6689-4e93-ab6a-58a56f374b83 admin admin] PATH_INFO: `/v3/projects/42380a38bfd847fc959e20c451671857/users/2709c92c6f9f46b886d72775cea7ae72/roles/44b64db484f24a46bf0b7ced46b519bf` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:48:51.597928 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e50ec2d-6689-4e93-ab6a-58a56f374b83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42380a38bfd847fc959e20c451671857, user_id=2709c92c6f9f46b886d72775cea7ae72, role_id=44b64db484f24a46bf0b7ced46b519bf)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:48:51.600620 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e50ec2d-6689-4e93-ab6a-58a56f374b83 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:48:51.681532 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1184/2379] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 08:48:51 2022] PUT /identity/v3/projects/42380a38bfd847fc959e20c451671857/users/2709c92c6f9f46b886d72775cea7ae72/roles/44b64db484f24a46bf0b7ced46b519bf => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:48:51.693329 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae94111-bd3a-48cb-aed3-826721fbeb27 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:48:51.693613 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae94111-bd3a-48cb-aed3-826721fbeb27 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:48:51.693887 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae94111-bd3a-48cb-aed3-826721fbeb27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:48:51.720780 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-dae94111-bd3a-48cb-aed3-826721fbeb27 None None] MFA Rules not processed for user `2709c92c6f9f46b886d72775cea7ae72`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:48:51.722029 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-dae94111-bd3a-48cb-aed3-826721fbeb27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:48:51.788973 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1196/2380] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:48:51 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:48:51.793609 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e50ec2d-6689-4e93-ab6a-58a56f374b83 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:48:51.840152 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:48:51.856785 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e50ec2d-6689-4e93-ab6a-58a56f374b83 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:48:51.862257 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db2f267-6732-4653-99cc-74c68ef7fce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:48:51.862863 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db2f267-6732-4653-99cc-74c68ef7fce9 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:48:51.863141 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db2f267-6732-4653-99cc-74c68ef7fce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:48:51.863409 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db2f267-6732-4653-99cc-74c68ef7fce9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:48:51.864042 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db2f267-6732-4653-99cc-74c68ef7fce9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:48:51.866001 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db2f267-6732-4653-99cc-74c68ef7fce9 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:48:51.936248 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1185/2381] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 08:48:51 2022] POST /identity/v3/projects => generated 381 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 08:48:51.937489 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae94111-bd3a-48cb-aed3-826721fbeb27 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:48:51.948562 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae94111-bd3a-48cb-aed3-826721fbeb27 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:48:52.007444 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce67130-5ab5-47bc-a0d5-9b43a884a08f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:48:52.008175 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce67130-5ab5-47bc-a0d5-9b43a884a08f admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:48:52.008464 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce67130-5ab5-47bc-a0d5-9b43a884a08f admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:48:52.008738 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce67130-5ab5-47bc-a0d5-9b43a884a08f admin admin] PATH_INFO: `/v3/users` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:48:52.009416 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce67130-5ab5-47bc-a0d5-9b43a884a08f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:48:52.011723 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce67130-5ab5-47bc-a0d5-9b43a884a08f admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:48:52.089424 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1197/2382] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 08:48:51 2022] POST /identity/v3/users => generated 297 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 08:48:52.090607 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db2f267-6732-4653-99cc-74c68ef7fce9 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:48:52.102042 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db2f267-6732-4653-99cc-74c68ef7fce9 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:48:52.106767 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9e66a9-e8d9-424a-a900-08e184ab3479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:48:52.107361 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9e66a9-e8d9-424a-a900-08e184ab3479 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:48:52.107682 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9e66a9-e8d9-424a-a900-08e184ab3479 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:48:52.107912 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9e66a9-e8d9-424a-a900-08e184ab3479 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:48:52.108551 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9e66a9-e8d9-424a-a900-08e184ab3479 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:48:52.110171 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9e66a9-e8d9-424a-a900-08e184ab3479 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:48:52.121096 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1186/2383] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:48:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:48:52.125409 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce67130-5ab5-47bc-a0d5-9b43a884a08f admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:48:52.136077 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce67130-5ab5-47bc-a0d5-9b43a884a08f admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:48:52.140743 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723db8ca-edeb-4f48-bba9-9616c58e7711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:48:52.141426 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723db8ca-edeb-4f48-bba9-9616c58e7711 admin admin] REQUEST_METHOD: `PUT` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:48:52.141702 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723db8ca-edeb-4f48-bba9-9616c58e7711 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:48:52.141976 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723db8ca-edeb-4f48-bba9-9616c58e7711 admin admin] PATH_INFO: `/v3/projects/174c7ed77fd14e2990109990229d58e7/users/22592e9058c54ff59b38c01f4131cd0a/roles/ec3751692d9c4ecf81b4a71c936c46be` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:48:52.157809 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723db8ca-edeb-4f48-bba9-9616c58e7711 admin admin] RBAC: Authorizing `identity:create_grant(project_id=174c7ed77fd14e2990109990229d58e7, user_id=22592e9058c54ff59b38c01f4131cd0a, role_id=ec3751692d9c4ecf81b4a71c936c46be)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:48:52.165978 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723db8ca-edeb-4f48-bba9-9616c58e7711 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:48:52.236345 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:48:52.242128 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1198/2384] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 08:48:52 2022] PUT /identity/v3/projects/174c7ed77fd14e2990109990229d58e7/users/22592e9058c54ff59b38c01f4131cd0a/roles/ec3751692d9c4ecf81b4a71c936c46be => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 08:48:52.246968 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9e66a9-e8d9-424a-a900-08e184ab3479 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:48:52.256701 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9e66a9-e8d9-424a-a900-08e184ab3479 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:48:52.316396 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d777ec31-c2f1-47e5-ba25-0b8368cd3225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:48:52.317038 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d777ec31-c2f1-47e5-ba25-0b8368cd3225 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:48:52.317366 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d777ec31-c2f1-47e5-ba25-0b8368cd3225 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:48:52.317638 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d777ec31-c2f1-47e5-ba25-0b8368cd3225 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:48:52.318305 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d777ec31-c2f1-47e5-ba25-0b8368cd3225 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:48:52.319881 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d777ec31-c2f1-47e5-ba25-0b8368cd3225 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:48:52.330769 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1187/2385] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:48:52 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:48:52.335224 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723db8ca-edeb-4f48-bba9-9616c58e7711 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:48:52.346111 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723db8ca-edeb-4f48-bba9-9616c58e7711 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:48:52.350837 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7732f3d4-348e-4f5e-87ef-8f56d2ee7e08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:48:52.351493 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7732f3d4-348e-4f5e-87ef-8f56d2ee7e08 admin admin] REQUEST_METHOD: `PUT` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:48:52.351768 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7732f3d4-348e-4f5e-87ef-8f56d2ee7e08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:48:52.352037 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7732f3d4-348e-4f5e-87ef-8f56d2ee7e08 admin admin] PATH_INFO: `/v3/projects/174c7ed77fd14e2990109990229d58e7/users/22592e9058c54ff59b38c01f4131cd0a/roles/ec3751692d9c4ecf81b4a71c936c46be` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:48:52.354039 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7732f3d4-348e-4f5e-87ef-8f56d2ee7e08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=174c7ed77fd14e2990109990229d58e7, user_id=22592e9058c54ff59b38c01f4131cd0a, role_id=ec3751692d9c4ecf81b4a71c936c46be)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:48:52.356722 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7732f3d4-348e-4f5e-87ef-8f56d2ee7e08 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:48:52.367095 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1199/2386] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 08:48:52 2022] PUT /identity/v3/projects/174c7ed77fd14e2990109990229d58e7/users/22592e9058c54ff59b38c01f4131cd0a/roles/ec3751692d9c4ecf81b4a71c936c46be => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:48:52.377253 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8680719d-c4de-4261-8daf-fd443e59a47a None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:48:52.385112 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8680719d-c4de-4261-8daf-fd443e59a47a None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:48:52.385112 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8680719d-c4de-4261-8daf-fd443e59a47a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:48:52.413531 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-8680719d-c4de-4261-8daf-fd443e59a47a None None] MFA Rules not processed for user `22592e9058c54ff59b38c01f4131cd0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:48:52.414825 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-8680719d-c4de-4261-8daf-fd443e59a47a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:48:52.485902 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1188/2387] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:48:52 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:48:52.498239 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7732f3d4-348e-4f5e-87ef-8f56d2ee7e08 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:48:52.500010 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-7732f3d4-348e-4f5e-87ef-8f56d2ee7e08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:48:52.567547 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7732f3d4-348e-4f5e-87ef-8f56d2ee7e08 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:48:52.571165 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024086ee-4442-43ca-8801-5a78f902f75c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ba74e28ced4cbba857f54222dfbe41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:48:52.571762 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024086ee-4442-43ca-8801-5a78f902f75c service cinder] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:48:52.572039 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024086ee-4442-43ca-8801-5a78f902f75c service cinder] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:48:52.572303 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024086ee-4442-43ca-8801-5a78f902f75c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:48:52.582456 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024086ee-4442-43ca-8801-5a78f902f75c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:48:52.583904 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024086ee-4442-43ca-8801-5a78f902f75c service cinder] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:48:52.593759 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1200/2388] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:48:52 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:48:52.725508 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8680719d-c4de-4261-8daf-fd443e59a47a None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:48:52.735506 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8680719d-c4de-4261-8daf-fd443e59a47a None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:48:52.740767 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24df3021-6639-4d5b-9a34-12e212adff2c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ba74e28ced4cbba857f54222dfbe41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:48:52.741557 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24df3021-6639-4d5b-9a34-12e212adff2c service cinder] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:48:52.741696 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24df3021-6639-4d5b-9a34-12e212adff2c service cinder] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:48:52.741874 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24df3021-6639-4d5b-9a34-12e212adff2c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:48:52.790989 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:48:52.811339 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24df3021-6639-4d5b-9a34-12e212adff2c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:48:52.812846 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24df3021-6639-4d5b-9a34-12e212adff2c service cinder] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:48:52.825852 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1189/2389] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:48:52 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 08:48:53.296296 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024086ee-4442-43ca-8801-5a78f902f75c service cinder] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:48:53.362298 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024086ee-4442-43ca-8801-5a78f902f75c service cinder] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:48:53.366913 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94462d88-410d-414b-ac77-278a21540050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:48:53.367543 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94462d88-410d-414b-ac77-278a21540050 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:48:53.367821 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94462d88-410d-414b-ac77-278a21540050 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:48:53.368090 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94462d88-410d-414b-ac77-278a21540050 admin admin] PATH_INFO: `/v3/users/2709c92c6f9f46b886d72775cea7ae72` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:48:53.369236 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94462d88-410d-414b-ac77-278a21540050 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2709c92c6f9f46b886d72775cea7ae72)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:48:53.371178 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94462d88-410d-414b-ac77-278a21540050 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:48:53.558685 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-94462d88-410d-414b-ac77-278a21540050 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2709c92c6f9f46b886d72775cea7ae72'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:48:53.571534 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-94462d88-410d-414b-ac77-278a21540050 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2709c92c6f9f46b886d72775cea7ae72'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:48:53.578406 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-94462d88-410d-414b-ac77-278a21540050 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2709c92c6f9f46b886d72775cea7ae72'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:48:53.646013 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-94462d88-410d-414b-ac77-278a21540050 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2709c92c6f9f46b886d72775cea7ae72'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:48:53.646710 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-94462d88-410d-414b-ac77-278a21540050 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2709c92c6f9f46b886d72775cea7ae72'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:48:53.653259 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1201/2390] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 08:48:53 2022] DELETE /identity/v3/users/2709c92c6f9f46b886d72775cea7ae72 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:48:53.657711 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e698b5-b85d-4af7-a6b9-32eb5afba6bb None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:48:53.658052 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e698b5-b85d-4af7-a6b9-32eb5afba6bb None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:48:53.658807 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e698b5-b85d-4af7-a6b9-32eb5afba6bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:48:53.679233 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-b7e698b5-b85d-4af7-a6b9-32eb5afba6bb None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:48:53.681053 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-b7e698b5-b85d-4af7-a6b9-32eb5afba6bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:48:53.742244 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1190/2391] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:48:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:48:53.751802 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94462d88-410d-414b-ac77-278a21540050 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:48:53.753796 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-94462d88-410d-414b-ac77-278a21540050 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:48:53.763817 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:48:53.813288 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94462d88-410d-414b-ac77-278a21540050 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:48:53.818270 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e38306c-3c56-4c47-9f3b-48b7917634de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:48:53.818880 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e38306c-3c56-4c47-9f3b-48b7917634de service neutron] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:48:53.819162 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e38306c-3c56-4c47-9f3b-48b7917634de service neutron] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:48:53.819435 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e38306c-3c56-4c47-9f3b-48b7917634de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:48:53.828479 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e38306c-3c56-4c47-9f3b-48b7917634de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:48:53.829970 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e38306c-3c56-4c47-9f3b-48b7917634de service neutron] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:48:53.839629 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1202/2392] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:48:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:48:54.207760 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e698b5-b85d-4af7-a6b9-32eb5afba6bb None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:48:54.209208 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-b7e698b5-b85d-4af7-a6b9-32eb5afba6bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:48:54.223425 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e698b5-b85d-4af7-a6b9-32eb5afba6bb None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:48:54.228189 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f03a11-8991-4a4e-a892-d55db8c18b0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:48:54.228690 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f03a11-8991-4a4e-a892-d55db8c18b0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:48:54.228992 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f03a11-8991-4a4e-a892-d55db8c18b0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:48:54.229741 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f03a11-8991-4a4e-a892-d55db8c18b0b admin admin] PATH_INFO: `/v3/projects/42380a38bfd847fc959e20c451671857` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:48:54.231034 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f03a11-8991-4a4e-a892-d55db8c18b0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=42380a38bfd847fc959e20c451671857)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:48:54.233152 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f03a11-8991-4a4e-a892-d55db8c18b0b admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:48:54.330159 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-d0f03a11-8991-4a4e-a892-d55db8c18b0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42380a38bfd847fc959e20c451671857', 'request_id': 'req-d0f03a11-8991-4a4e-a892-d55db8c18b0b'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:48:54.406278 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-d0f03a11-8991-4a4e-a892-d55db8c18b0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42380a38bfd847fc959e20c451671857', 'request_id': 'req-d0f03a11-8991-4a4e-a892-d55db8c18b0b'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:48:54.413125 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-d0f03a11-8991-4a4e-a892-d55db8c18b0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42380a38bfd847fc959e20c451671857', 'request_id': 'req-d0f03a11-8991-4a4e-a892-d55db8c18b0b'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:48:54.424862 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-d0f03a11-8991-4a4e-a892-d55db8c18b0b admin admin] The token cache is being invalidate because project 42380a38bfd847fc959e20c451671857 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 08:48:54.425181 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-d0f03a11-8991-4a4e-a892-d55db8c18b0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:48:54.425817 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-d0f03a11-8991-4a4e-a892-d55db8c18b0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:48:54.427659 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1191/2393] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 08:48:54 2022] DELETE /identity/v3/projects/42380a38bfd847fc959e20c451671857 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:48:54.435703 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e38306c-3c56-4c47-9f3b-48b7917634de service neutron] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:48:54.445902 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e38306c-3c56-4c47-9f3b-48b7917634de service neutron] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:48:54.503466 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc07b1c-6571-4bf7-a576-e0fcd7903745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:48:54.504100 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc07b1c-6571-4bf7-a576-e0fcd7903745 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:48:54.504817 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc07b1c-6571-4bf7-a576-e0fcd7903745 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:48:54.504817 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc07b1c-6571-4bf7-a576-e0fcd7903745 admin admin] PATH_INFO: `/v3/users/22592e9058c54ff59b38c01f4131cd0a` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:48:54.508644 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc07b1c-6571-4bf7-a576-e0fcd7903745 admin admin] RBAC: Authorizing `identity:delete_user(user_id=22592e9058c54ff59b38c01f4131cd0a)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:48:54.511008 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc07b1c-6571-4bf7-a576-e0fcd7903745 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:48:54.739120 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-dcc07b1c-6571-4bf7-a576-e0fcd7903745 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22592e9058c54ff59b38c01f4131cd0a'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:48:54.750955 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-dcc07b1c-6571-4bf7-a576-e0fcd7903745 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22592e9058c54ff59b38c01f4131cd0a'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:48:54.758272 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-dcc07b1c-6571-4bf7-a576-e0fcd7903745 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22592e9058c54ff59b38c01f4131cd0a'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:48:54.817434 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-dcc07b1c-6571-4bf7-a576-e0fcd7903745 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22592e9058c54ff59b38c01f4131cd0a'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:48:54.818058 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-dcc07b1c-6571-4bf7-a576-e0fcd7903745 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22592e9058c54ff59b38c01f4131cd0a'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:48:54.824265 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1203/2394] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 08:48:54 2022] DELETE /identity/v3/users/22592e9058c54ff59b38c01f4131cd0a => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:48:55.247651 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f03a11-8991-4a4e-a892-d55db8c18b0b admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:48:55.249221 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-d0f03a11-8991-4a4e-a892-d55db8c18b0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:48:55.265110 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f03a11-8991-4a4e-a892-d55db8c18b0b admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:48:55.322702 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176ff923-6255-413a-bc26-f914638e47af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:48:55.323327 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176ff923-6255-413a-bc26-f914638e47af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:48:55.323608 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176ff923-6255-413a-bc26-f914638e47af admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:48:55.323877 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176ff923-6255-413a-bc26-f914638e47af admin admin] PATH_INFO: `/v3/projects/174c7ed77fd14e2990109990229d58e7` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:48:55.325003 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176ff923-6255-413a-bc26-f914638e47af admin admin] RBAC: Authorizing `identity:delete_project(project_id=174c7ed77fd14e2990109990229d58e7)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:48:55.326919 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176ff923-6255-413a-bc26-f914638e47af admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:48:55.329151 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:48:55.424170 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-176ff923-6255-413a-bc26-f914638e47af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '174c7ed77fd14e2990109990229d58e7', 'request_id': 'req-176ff923-6255-413a-bc26-f914638e47af'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:48:55.497462 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-176ff923-6255-413a-bc26-f914638e47af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '174c7ed77fd14e2990109990229d58e7', 'request_id': 'req-176ff923-6255-413a-bc26-f914638e47af'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:48:55.503867 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-176ff923-6255-413a-bc26-f914638e47af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '174c7ed77fd14e2990109990229d58e7', 'request_id': 'req-176ff923-6255-413a-bc26-f914638e47af'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:48:55.515099 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-176ff923-6255-413a-bc26-f914638e47af admin admin] The token cache is being invalidate because project 174c7ed77fd14e2990109990229d58e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 08:48:55.515386 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-176ff923-6255-413a-bc26-f914638e47af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:48:55.516014 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-176ff923-6255-413a-bc26-f914638e47af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:48:55.517581 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1192/2395] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 08:48:55 2022] DELETE /identity/v3/projects/174c7ed77fd14e2990109990229d58e7 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:48:55.530005 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61220a2d-fb3a-4000-948e-8adbbd4575a8 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:48:55.530563 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61220a2d-fb3a-4000-948e-8adbbd4575a8 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:48:55.530563 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61220a2d-fb3a-4000-948e-8adbbd4575a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:48:55.551511 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-61220a2d-fb3a-4000-948e-8adbbd4575a8 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:48:55.552810 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-61220a2d-fb3a-4000-948e-8adbbd4575a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:48:55.577869 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:48:55.613027 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1204/2396] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:48:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:48:55.620779 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cf1dd8-66f3-4c6f-bbdc-02e02e86e403 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:48:55.621088 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cf1dd8-66f3-4c6f-bbdc-02e02e86e403 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:48:55.621359 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cf1dd8-66f3-4c6f-bbdc-02e02e86e403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:48:55.642396 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-d9cf1dd8-66f3-4c6f-bbdc-02e02e86e403 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:48:55.643711 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-d9cf1dd8-66f3-4c6f-bbdc-02e02e86e403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:48:55.651610 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1193/2397] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:48:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:48:55.658730 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61220a2d-fb3a-4000-948e-8adbbd4575a8 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:48:55.665745 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61220a2d-fb3a-4000-948e-8adbbd4575a8 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:48:55.670337 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef013a7-5b74-4249-9cf4-42ab6714c396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:48:55.670932 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef013a7-5b74-4249-9cf4-42ab6714c396 admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:48:55.671210 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef013a7-5b74-4249-9cf4-42ab6714c396 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:48:55.671478 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef013a7-5b74-4249-9cf4-42ab6714c396 admin admin] PATH_INFO: `/v3/domains` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:48:55.672119 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef013a7-5b74-4249-9cf4-42ab6714c396 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:48:55.672420 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef013a7-5b74-4249-9cf4-42ab6714c396 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:48:55.674819 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef013a7-5b74-4249-9cf4-42ab6714c396 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:48:55.686502 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1205/2398] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Aug 10 08:48:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:48:55.690837 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cf1dd8-66f3-4c6f-bbdc-02e02e86e403 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:48:55.699984 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cf1dd8-66f3-4c6f-bbdc-02e02e86e403 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:48:55.704680 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3c1fac-3e65-457f-9ed1-2d9fddf2f037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:48:55.705312 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3c1fac-3e65-457f-9ed1-2d9fddf2f037 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:48:55.705665 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3c1fac-3e65-457f-9ed1-2d9fddf2f037 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:48:55.705935 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3c1fac-3e65-457f-9ed1-2d9fddf2f037 admin admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:48:55.706552 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3c1fac-3e65-457f-9ed1-2d9fddf2f037 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:48:55.708426 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3c1fac-3e65-457f-9ed1-2d9fddf2f037 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:48:55.790842 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1194/2399] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 08:48:55 2022] POST /identity/v3/projects => generated 385 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:48:55.794883 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef013a7-5b74-4249-9cf4-42ab6714c396 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:48:55.805778 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef013a7-5b74-4249-9cf4-42ab6714c396 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:48:55.863439 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d20b42-c6b1-4ebe-b995-7fd3855faac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:48:55.864047 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d20b42-c6b1-4ebe-b995-7fd3855faac6 admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:48:55.864321 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d20b42-c6b1-4ebe-b995-7fd3855faac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:48:55.864587 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d20b42-c6b1-4ebe-b995-7fd3855faac6 admin admin] PATH_INFO: `/v3/users` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:48:55.865236 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d20b42-c6b1-4ebe-b995-7fd3855faac6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:48:55.867132 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d20b42-c6b1-4ebe-b995-7fd3855faac6 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:48:55.948807 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1206/2400] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 08:48:55 2022] POST /identity/v3/users => generated 293 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 08:48:55.952853 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3c1fac-3e65-457f-9ed1-2d9fddf2f037 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:48:55.963619 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3c1fac-3e65-457f-9ed1-2d9fddf2f037 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:48:55.968198 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3b7a91-3e05-4f2d-8de6-cafa16970e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:48:55.968787 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3b7a91-3e05-4f2d-8de6-cafa16970e4a admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:48:55.969096 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3b7a91-3e05-4f2d-8de6-cafa16970e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:48:55.969369 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3b7a91-3e05-4f2d-8de6-cafa16970e4a admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:48:55.970008 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3b7a91-3e05-4f2d-8de6-cafa16970e4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:48:55.972349 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3b7a91-3e05-4f2d-8de6-cafa16970e4a admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:48:55.982602 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1195/2401] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:48:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:48:55.986461 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d20b42-c6b1-4ebe-b995-7fd3855faac6 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:48:55.996609 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d20b42-c6b1-4ebe-b995-7fd3855faac6 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:48:56.001323 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55365f5-9b7a-4fd3-be5b-ef06efefa10d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:48:56.001975 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55365f5-9b7a-4fd3-be5b-ef06efefa10d admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:48:56.002260 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55365f5-9b7a-4fd3-be5b-ef06efefa10d admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:48:56.002542 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55365f5-9b7a-4fd3-be5b-ef06efefa10d admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:48:56.003183 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55365f5-9b7a-4fd3-be5b-ef06efefa10d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:48:56.004892 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55365f5-9b7a-4fd3-be5b-ef06efefa10d admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:48:56.015725 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1207/2402] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:48:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:48:56.020338 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3b7a91-3e05-4f2d-8de6-cafa16970e4a admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:48:56.030217 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3b7a91-3e05-4f2d-8de6-cafa16970e4a admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:48:56.035431 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d877c1-58d6-4680-8938-1f208935ab0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:48:56.036065 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d877c1-58d6-4680-8938-1f208935ab0b admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:48:56.036343 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d877c1-58d6-4680-8938-1f208935ab0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:48:56.036612 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d877c1-58d6-4680-8938-1f208935ab0b admin admin] PATH_INFO: `/v3/projects/e98254452fc647cf8b9c7cc11845801d/users/75b5201e6df64288956e99b217e35ad1/roles/44b64db484f24a46bf0b7ced46b519bf` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:48:56.051032 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d877c1-58d6-4680-8938-1f208935ab0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e98254452fc647cf8b9c7cc11845801d, user_id=75b5201e6df64288956e99b217e35ad1, role_id=44b64db484f24a46bf0b7ced46b519bf)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:48:56.053643 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d877c1-58d6-4680-8938-1f208935ab0b admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:48:56.154223 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1196/2403] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 08:48:56 2022] PUT /identity/v3/projects/e98254452fc647cf8b9c7cc11845801d/users/75b5201e6df64288956e99b217e35ad1/roles/44b64db484f24a46bf0b7ced46b519bf => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:48:56.166161 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d6c6aa-3af2-4181-be41-c50f47eccc27 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:48:56.166452 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d6c6aa-3af2-4181-be41-c50f47eccc27 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:48:56.166730 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d6c6aa-3af2-4181-be41-c50f47eccc27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:48:56.194521 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-64d6c6aa-3af2-4181-be41-c50f47eccc27 None None] MFA Rules not processed for user `75b5201e6df64288956e99b217e35ad1`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:48:56.195761 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-64d6c6aa-3af2-4181-be41-c50f47eccc27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:48:56.260388 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1208/2404] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:48:56 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:48:56.271682 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d877c1-58d6-4680-8938-1f208935ab0b admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:48:56.273361 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-08d877c1-58d6-4680-8938-1f208935ab0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:48:56.329261 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d877c1-58d6-4680-8938-1f208935ab0b admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:48:56.333445 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0550589f-2dc9-4415-a5a1-d2ae24134107 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ba74e28ced4cbba857f54222dfbe41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:48:56.334042 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0550589f-2dc9-4415-a5a1-d2ae24134107 service cinder] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:48:56.334318 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0550589f-2dc9-4415-a5a1-d2ae24134107 service cinder] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:48:56.334582 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0550589f-2dc9-4415-a5a1-d2ae24134107 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:48:56.342732 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0550589f-2dc9-4415-a5a1-d2ae24134107 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:48:56.344146 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0550589f-2dc9-4415-a5a1-d2ae24134107 service cinder] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:48:56.352754 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1197/2405] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:48:56 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:49:06.257379 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=86,keystone:INSERT=3 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:49:06.348879 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=76,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:49:08.344087 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d6c6aa-3af2-4181-be41-c50f47eccc27 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:49:08.407126 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d6c6aa-3af2-4181-be41-c50f47eccc27 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:49:08.412135 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407b0590-9792-4fd7-818b-48f1c50387f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:49:08.412748 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407b0590-9792-4fd7-818b-48f1c50387f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:49:08.413059 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407b0590-9792-4fd7-818b-48f1c50387f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:49:08.413332 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407b0590-9792-4fd7-818b-48f1c50387f7 admin admin] PATH_INFO: `/v3/users/75b5201e6df64288956e99b217e35ad1` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:49:08.414437 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407b0590-9792-4fd7-818b-48f1c50387f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75b5201e6df64288956e99b217e35ad1)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:49:08.416353 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407b0590-9792-4fd7-818b-48f1c50387f7 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:49:08.596764 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-407b0590-9792-4fd7-818b-48f1c50387f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75b5201e6df64288956e99b217e35ad1'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:49:08.608891 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-407b0590-9792-4fd7-818b-48f1c50387f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75b5201e6df64288956e99b217e35ad1'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:49:08.616715 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-407b0590-9792-4fd7-818b-48f1c50387f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75b5201e6df64288956e99b217e35ad1'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:49:08.696898 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-407b0590-9792-4fd7-818b-48f1c50387f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75b5201e6df64288956e99b217e35ad1'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:49:08.697570 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-407b0590-9792-4fd7-818b-48f1c50387f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75b5201e6df64288956e99b217e35ad1'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:49:08.703572 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1209/2406] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 08:49:08 2022] DELETE /identity/v3/users/75b5201e6df64288956e99b217e35ad1 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:49:08.711144 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87f04c9-b1b2-4463-8b0a-615ed74e1899 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:49:08.711619 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87f04c9-b1b2-4463-8b0a-615ed74e1899 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:49:08.711724 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87f04c9-b1b2-4463-8b0a-615ed74e1899 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:49:08.730106 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-a87f04c9-b1b2-4463-8b0a-615ed74e1899 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:49:08.731343 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-a87f04c9-b1b2-4463-8b0a-615ed74e1899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:49:08.792440 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1198/2407] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:49:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:49:08.804022 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407b0590-9792-4fd7-818b-48f1c50387f7 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:49:08.805625 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-407b0590-9792-4fd7-818b-48f1c50387f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:49:08.864729 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407b0590-9792-4fd7-818b-48f1c50387f7 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:49:08.868239 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e58cfa-3651-4083-bcc2-aa96b3c10775 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:49:08.868830 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e58cfa-3651-4083-bcc2-aa96b3c10775 service neutron] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:49:08.869133 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e58cfa-3651-4083-bcc2-aa96b3c10775 service neutron] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:49:08.869400 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e58cfa-3651-4083-bcc2-aa96b3c10775 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:49:08.878077 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e58cfa-3651-4083-bcc2-aa96b3c10775 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:49:08.880642 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e58cfa-3651-4083-bcc2-aa96b3c10775 service neutron] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:49:08.890098 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1210/2408] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:49:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:49:09.276115 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87f04c9-b1b2-4463-8b0a-615ed74e1899 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:49:09.279393 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-a87f04c9-b1b2-4463-8b0a-615ed74e1899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:49:09.290493 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87f04c9-b1b2-4463-8b0a-615ed74e1899 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:49:09.295328 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f5b0e2-cc47-4e58-8d36-d558de976b04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:49:09.296043 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f5b0e2-cc47-4e58-8d36-d558de976b04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:49:09.296324 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f5b0e2-cc47-4e58-8d36-d558de976b04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:49:09.296598 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f5b0e2-cc47-4e58-8d36-d558de976b04 admin admin] PATH_INFO: `/v3/projects/e98254452fc647cf8b9c7cc11845801d` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:49:09.297752 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f5b0e2-cc47-4e58-8d36-d558de976b04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e98254452fc647cf8b9c7cc11845801d)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:49:09.299661 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f5b0e2-cc47-4e58-8d36-d558de976b04 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:49:09.383182 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-30f5b0e2-cc47-4e58-8d36-d558de976b04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e98254452fc647cf8b9c7cc11845801d', 'request_id': 'req-30f5b0e2-cc47-4e58-8d36-d558de976b04'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:49:09.445937 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-30f5b0e2-cc47-4e58-8d36-d558de976b04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e98254452fc647cf8b9c7cc11845801d', 'request_id': 'req-30f5b0e2-cc47-4e58-8d36-d558de976b04'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:49:09.452424 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-30f5b0e2-cc47-4e58-8d36-d558de976b04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e98254452fc647cf8b9c7cc11845801d', 'request_id': 'req-30f5b0e2-cc47-4e58-8d36-d558de976b04'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:49:09.463758 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-30f5b0e2-cc47-4e58-8d36-d558de976b04 admin admin] The token cache is being invalidate because project e98254452fc647cf8b9c7cc11845801d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 08:49:09.464047 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-30f5b0e2-cc47-4e58-8d36-d558de976b04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:49:09.464685 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-30f5b0e2-cc47-4e58-8d36-d558de976b04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:49:09.466234 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1199/2409] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 08:49:09 2022] DELETE /identity/v3/projects/e98254452fc647cf8b9c7cc11845801d => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:49:09.471145 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd4e5af-d144-4af8-9eb3-22718dca8779 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:49:09.471617 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd4e5af-d144-4af8-9eb3-22718dca8779 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:49:09.471714 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd4e5af-d144-4af8-9eb3-22718dca8779 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:49:09.491702 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-7fd4e5af-d144-4af8-9eb3-22718dca8779 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:49:09.492894 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-7fd4e5af-d144-4af8-9eb3-22718dca8779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:49:09.509864 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:49:09.550009 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1211/2410] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:49:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:49:09.558538 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22319e5b-c724-4c15-b6cc-23e4dc539321 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:49:09.558855 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22319e5b-c724-4c15-b6cc-23e4dc539321 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:49:09.559128 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22319e5b-c724-4c15-b6cc-23e4dc539321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:49:09.579407 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-22319e5b-c724-4c15-b6cc-23e4dc539321 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:49:09.580591 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-22319e5b-c724-4c15-b6cc-23e4dc539321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:49:09.589822 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1200/2411] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:49:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:49:09.594095 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd4e5af-d144-4af8-9eb3-22718dca8779 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:49:09.604056 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd4e5af-d144-4af8-9eb3-22718dca8779 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:49:09.608546 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b18dcb1-00df-43cd-b661-a8a99788518c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:49:09.609179 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b18dcb1-00df-43cd-b661-a8a99788518c admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:49:09.609458 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b18dcb1-00df-43cd-b661-a8a99788518c admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:49:09.609724 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b18dcb1-00df-43cd-b661-a8a99788518c admin admin] PATH_INFO: `/v3/domains` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:49:09.610355 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b18dcb1-00df-43cd-b661-a8a99788518c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:49:09.610652 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b18dcb1-00df-43cd-b661-a8a99788518c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:49:09.612224 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b18dcb1-00df-43cd-b661-a8a99788518c admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:49:09.624993 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1212/2412] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Aug 10 08:49:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:49:09.629841 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22319e5b-c724-4c15-b6cc-23e4dc539321 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:49:09.639515 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22319e5b-c724-4c15-b6cc-23e4dc539321 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:49:09.644211 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047e9676-960e-4b5b-996a-cc2c4eb224a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:49:09.644996 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047e9676-960e-4b5b-996a-cc2c4eb224a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:49:09.645408 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047e9676-960e-4b5b-996a-cc2c4eb224a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:49:09.645677 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047e9676-960e-4b5b-996a-cc2c4eb224a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:49:09.646300 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047e9676-960e-4b5b-996a-cc2c4eb224a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:49:09.648174 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047e9676-960e-4b5b-996a-cc2c4eb224a0 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:49:09.732626 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1201/2413] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 08:49:09 2022] POST /identity/v3/projects => generated 359 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:49:09.736804 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b18dcb1-00df-43cd-b661-a8a99788518c admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:49:09.747746 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b18dcb1-00df-43cd-b661-a8a99788518c admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:49:09.804378 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8e2943-b4cf-4383-a479-b08221dd128d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:49:09.805019 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8e2943-b4cf-4383-a479-b08221dd128d admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:49:09.805304 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8e2943-b4cf-4383-a479-b08221dd128d admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:49:09.805576 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8e2943-b4cf-4383-a479-b08221dd128d admin admin] PATH_INFO: `/v3/users` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:49:09.806201 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8e2943-b4cf-4383-a479-b08221dd128d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:49:09.808114 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8e2943-b4cf-4383-a479-b08221dd128d admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:49:09.879946 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1213/2414] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 08:49:09 2022] POST /identity/v3/users => generated 280 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:49:09.884097 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047e9676-960e-4b5b-996a-cc2c4eb224a0 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:49:09.895212 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047e9676-960e-4b5b-996a-cc2c4eb224a0 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:49:09.899769 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078a1104-b8ec-4e02-9da5-aa83538559c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:49:09.900361 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078a1104-b8ec-4e02-9da5-aa83538559c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:49:09.900666 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078a1104-b8ec-4e02-9da5-aa83538559c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:49:09.900965 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078a1104-b8ec-4e02-9da5-aa83538559c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:49:09.901630 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078a1104-b8ec-4e02-9da5-aa83538559c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:49:09.903275 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078a1104-b8ec-4e02-9da5-aa83538559c0 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:49:09.913757 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1202/2415] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:49:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:49:09.917553 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8e2943-b4cf-4383-a479-b08221dd128d admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:49:09.927933 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8e2943-b4cf-4383-a479-b08221dd128d admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:49:09.932415 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8309798-b6a1-4cda-868f-f5090f901543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:49:09.933042 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8309798-b6a1-4cda-868f-f5090f901543 admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:49:09.933319 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8309798-b6a1-4cda-868f-f5090f901543 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:49:09.933587 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8309798-b6a1-4cda-868f-f5090f901543 admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:49:09.934215 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8309798-b6a1-4cda-868f-f5090f901543 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:49:09.935779 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8309798-b6a1-4cda-868f-f5090f901543 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:49:09.946202 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1214/2416] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:49:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:49:09.950172 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078a1104-b8ec-4e02-9da5-aa83538559c0 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:49:09.960183 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078a1104-b8ec-4e02-9da5-aa83538559c0 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:49:09.964699 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c6e57a-537f-44c3-a497-e23af51a4550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:49:09.965373 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c6e57a-537f-44c3-a497-e23af51a4550 admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:49:09.965654 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c6e57a-537f-44c3-a497-e23af51a4550 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:49:09.965927 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c6e57a-537f-44c3-a497-e23af51a4550 admin admin] PATH_INFO: `/v3/projects/7c626115b9c74f7ebeccca05d523e313/users/0fdcb699b9ff4835b2d3e31e4b896d55/roles/44b64db484f24a46bf0b7ced46b519bf` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:49:09.980581 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c6e57a-537f-44c3-a497-e23af51a4550 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c626115b9c74f7ebeccca05d523e313, user_id=0fdcb699b9ff4835b2d3e31e4b896d55, role_id=44b64db484f24a46bf0b7ced46b519bf)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:49:09.983305 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c6e57a-537f-44c3-a497-e23af51a4550 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:49:10.045853 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1203/2417] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 08:49:09 2022] PUT /identity/v3/projects/7c626115b9c74f7ebeccca05d523e313/users/0fdcb699b9ff4835b2d3e31e4b896d55/roles/44b64db484f24a46bf0b7ced46b519bf => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:49:10.059877 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf06b9d-cabc-4216-9c8b-7c848c216dae None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:49:10.060380 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf06b9d-cabc-4216-9c8b-7c848c216dae None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:49:10.060744 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf06b9d-cabc-4216-9c8b-7c848c216dae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:49:10.088506 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-faf06b9d-cabc-4216-9c8b-7c848c216dae None None] MFA Rules not processed for user `0fdcb699b9ff4835b2d3e31e4b896d55`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:49:10.089738 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-faf06b9d-cabc-4216-9c8b-7c848c216dae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:49:10.154017 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1215/2418] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:49:10 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:49:10.187052 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c6e57a-537f-44c3-a497-e23af51a4550 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:49:10.188742 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-35c6e57a-537f-44c3-a497-e23af51a4550 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:49:10.211203 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:49:10.246083 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c6e57a-537f-44c3-a497-e23af51a4550 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:49:10.249740 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6760f12-08ae-43c6-a178-eb86553b4b23 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ba74e28ced4cbba857f54222dfbe41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:49:10.250341 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6760f12-08ae-43c6-a178-eb86553b4b23 service cinder] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:49:10.250622 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6760f12-08ae-43c6-a178-eb86553b4b23 service cinder] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:49:10.250893 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6760f12-08ae-43c6-a178-eb86553b4b23 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:49:10.259382 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6760f12-08ae-43c6-a178-eb86553b4b23 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:49:10.261140 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6760f12-08ae-43c6-a178-eb86553b4b23 service cinder] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:49:10.271075 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1204/2419] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:49:10 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:49:10.286731 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf06b9d-cabc-4216-9c8b-7c848c216dae None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:49:10.305960 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:49:10.349454 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf06b9d-cabc-4216-9c8b-7c848c216dae None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:49:10.354364 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a5b5bf-08b2-40e7-9192-d8461993e79b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:49:10.354998 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a5b5bf-08b2-40e7-9192-d8461993e79b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:49:10.355411 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a5b5bf-08b2-40e7-9192-d8461993e79b admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:49:10.356044 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a5b5bf-08b2-40e7-9192-d8461993e79b admin admin] PATH_INFO: `/v3/users/0fdcb699b9ff4835b2d3e31e4b896d55` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:49:10.357258 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a5b5bf-08b2-40e7-9192-d8461993e79b admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fdcb699b9ff4835b2d3e31e4b896d55)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:49:10.359307 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a5b5bf-08b2-40e7-9192-d8461993e79b admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:49:10.561819 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-d7a5b5bf-08b2-40e7-9192-d8461993e79b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fdcb699b9ff4835b2d3e31e4b896d55'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:49:10.573842 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-d7a5b5bf-08b2-40e7-9192-d8461993e79b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fdcb699b9ff4835b2d3e31e4b896d55'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:49:10.580407 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-d7a5b5bf-08b2-40e7-9192-d8461993e79b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fdcb699b9ff4835b2d3e31e4b896d55'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:49:10.628593 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-d7a5b5bf-08b2-40e7-9192-d8461993e79b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fdcb699b9ff4835b2d3e31e4b896d55'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:49:10.629711 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-d7a5b5bf-08b2-40e7-9192-d8461993e79b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fdcb699b9ff4835b2d3e31e4b896d55'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:49:10.635899 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1216/2420] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 08:49:10 2022] DELETE /identity/v3/users/0fdcb699b9ff4835b2d3e31e4b896d55 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:49:10.640295 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e70500-1d9e-4c31-b3a6-99e8d0de44e5 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:49:10.640588 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e70500-1d9e-4c31-b3a6-99e8d0de44e5 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:49:10.640861 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e70500-1d9e-4c31-b3a6-99e8d0de44e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:49:10.660141 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-30e70500-1d9e-4c31-b3a6-99e8d0de44e5 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:49:10.661757 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-30e70500-1d9e-4c31-b3a6-99e8d0de44e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:49:10.720895 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1205/2421] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:49:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:49:10.730648 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a5b5bf-08b2-40e7-9192-d8461993e79b admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:49:10.732063 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-d7a5b5bf-08b2-40e7-9192-d8461993e79b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:49:10.794063 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a5b5bf-08b2-40e7-9192-d8461993e79b admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:49:10.798065 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2877fa04-338d-4bd7-8cf3-dc07a1bd5490 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:49:10.798659 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2877fa04-338d-4bd7-8cf3-dc07a1bd5490 service neutron] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:49:10.798937 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2877fa04-338d-4bd7-8cf3-dc07a1bd5490 service neutron] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:49:10.799206 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2877fa04-338d-4bd7-8cf3-dc07a1bd5490 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:49:10.808509 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2877fa04-338d-4bd7-8cf3-dc07a1bd5490 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:49:10.809973 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2877fa04-338d-4bd7-8cf3-dc07a1bd5490 service neutron] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:49:10.819844 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1217/2422] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:49:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:49:11.207877 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e70500-1d9e-4c31-b3a6-99e8d0de44e5 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:49:11.209319 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-30e70500-1d9e-4c31-b3a6-99e8d0de44e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:49:11.226126 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e70500-1d9e-4c31-b3a6-99e8d0de44e5 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:49:11.231021 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc7e4e6-f0c0-4c01-a078-158dff676aff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:49:11.235696 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc7e4e6-f0c0-4c01-a078-158dff676aff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:49:11.235696 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc7e4e6-f0c0-4c01-a078-158dff676aff admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:49:11.235696 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc7e4e6-f0c0-4c01-a078-158dff676aff admin admin] PATH_INFO: `/v3/projects/7c626115b9c74f7ebeccca05d523e313` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:49:11.235696 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc7e4e6-f0c0-4c01-a078-158dff676aff admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c626115b9c74f7ebeccca05d523e313)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:49:11.235696 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc7e4e6-f0c0-4c01-a078-158dff676aff admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:49:11.331328 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-0fc7e4e6-f0c0-4c01-a078-158dff676aff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c626115b9c74f7ebeccca05d523e313', 'request_id': 'req-0fc7e4e6-f0c0-4c01-a078-158dff676aff'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:49:11.383102 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-0fc7e4e6-f0c0-4c01-a078-158dff676aff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c626115b9c74f7ebeccca05d523e313', 'request_id': 'req-0fc7e4e6-f0c0-4c01-a078-158dff676aff'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:49:11.389363 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-0fc7e4e6-f0c0-4c01-a078-158dff676aff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c626115b9c74f7ebeccca05d523e313', 'request_id': 'req-0fc7e4e6-f0c0-4c01-a078-158dff676aff'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:49:11.401528 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-0fc7e4e6-f0c0-4c01-a078-158dff676aff admin admin] The token cache is being invalidate because project 7c626115b9c74f7ebeccca05d523e313 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 08:49:11.401941 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-0fc7e4e6-f0c0-4c01-a078-158dff676aff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:49:11.402984 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-0fc7e4e6-f0c0-4c01-a078-158dff676aff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:49:11.404413 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1206/2423] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 08:49:11 2022] DELETE /identity/v3/projects/7c626115b9c74f7ebeccca05d523e313 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:49:11.409476 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6facf7b8-d4f3-4f44-a42b-4cf95edfc07e None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:49:11.409773 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6facf7b8-d4f3-4f44-a42b-4cf95edfc07e None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:49:11.410049 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6facf7b8-d4f3-4f44-a42b-4cf95edfc07e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:49:11.430603 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-6facf7b8-d4f3-4f44-a42b-4cf95edfc07e None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:49:11.431797 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-6facf7b8-d4f3-4f44-a42b-4cf95edfc07e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:49:11.464530 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:49:11.489642 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1218/2424] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:49:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:49:11.497450 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2642e2dc-ab0a-4327-b07c-75de46ebc920 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:49:11.497734 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2642e2dc-ab0a-4327-b07c-75de46ebc920 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:49:11.505802 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2642e2dc-ab0a-4327-b07c-75de46ebc920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:49:11.518303 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-2642e2dc-ab0a-4327-b07c-75de46ebc920 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:49:11.519493 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-2642e2dc-ab0a-4327-b07c-75de46ebc920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:49:11.528005 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1207/2425] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:49:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:49:11.532137 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6facf7b8-d4f3-4f44-a42b-4cf95edfc07e None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:49:11.542089 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6facf7b8-d4f3-4f44-a42b-4cf95edfc07e None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:49:11.546580 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5670622-0365-4989-a1f4-8e667518384b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:49:11.547166 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5670622-0365-4989-a1f4-8e667518384b admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:49:11.547442 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5670622-0365-4989-a1f4-8e667518384b admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:49:11.547706 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5670622-0365-4989-a1f4-8e667518384b admin admin] PATH_INFO: `/v3/domains` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:49:11.548352 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5670622-0365-4989-a1f4-8e667518384b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:49:11.548846 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5670622-0365-4989-a1f4-8e667518384b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:49:11.550787 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5670622-0365-4989-a1f4-8e667518384b admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:49:11.563384 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1219/2426] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Aug 10 08:49:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:49:11.567601 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2642e2dc-ab0a-4327-b07c-75de46ebc920 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:49:11.577434 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2642e2dc-ab0a-4327-b07c-75de46ebc920 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:49:11.582011 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc801c63-e383-4eb4-a499-77cf73c9d6b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:49:11.583097 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc801c63-e383-4eb4-a499-77cf73c9d6b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:49:11.583372 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc801c63-e383-4eb4-a499-77cf73c9d6b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:49:11.583642 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc801c63-e383-4eb4-a499-77cf73c9d6b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:49:11.584270 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc801c63-e383-4eb4-a499-77cf73c9d6b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:49:11.586181 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc801c63-e383-4eb4-a499-77cf73c9d6b7 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:49:11.669466 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1208/2427] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 08:49:11 2022] POST /identity/v3/projects => generated 365 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 08:49:11.673504 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5670622-0365-4989-a1f4-8e667518384b admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:49:11.684187 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5670622-0365-4989-a1f4-8e667518384b admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:49:11.741598 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5107adbd-077f-4db7-8026-5fb8defbc311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:49:11.742192 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5107adbd-077f-4db7-8026-5fb8defbc311 admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:49:11.742466 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5107adbd-077f-4db7-8026-5fb8defbc311 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:49:11.742732 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5107adbd-077f-4db7-8026-5fb8defbc311 admin admin] PATH_INFO: `/v3/users` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:49:11.743349 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5107adbd-077f-4db7-8026-5fb8defbc311 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:49:11.745285 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5107adbd-077f-4db7-8026-5fb8defbc311 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:49:11.828978 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1220/2428] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 08:49:11 2022] POST /identity/v3/users => generated 283 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:49:11.833326 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc801c63-e383-4eb4-a499-77cf73c9d6b7 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:49:11.843828 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc801c63-e383-4eb4-a499-77cf73c9d6b7 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:49:11.848700 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08e067f-75ac-445c-910a-81eeba9c67d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:49:11.849332 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08e067f-75ac-445c-910a-81eeba9c67d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:49:11.849613 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08e067f-75ac-445c-910a-81eeba9c67d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:49:11.849886 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08e067f-75ac-445c-910a-81eeba9c67d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:49:11.850525 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08e067f-75ac-445c-910a-81eeba9c67d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:49:11.852107 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08e067f-75ac-445c-910a-81eeba9c67d2 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:49:11.861998 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1209/2429] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:49:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:49:11.866849 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5107adbd-077f-4db7-8026-5fb8defbc311 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:49:11.876765 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5107adbd-077f-4db7-8026-5fb8defbc311 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:49:11.882222 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3771b8e-7f01-4976-ab56-0297877d9b7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:49:11.882809 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3771b8e-7f01-4976-ab56-0297877d9b7e admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:49:11.883089 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3771b8e-7f01-4976-ab56-0297877d9b7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:49:11.883360 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3771b8e-7f01-4976-ab56-0297877d9b7e admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:49:11.883987 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3771b8e-7f01-4976-ab56-0297877d9b7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:49:11.885876 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3771b8e-7f01-4976-ab56-0297877d9b7e admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:49:11.896552 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1221/2430] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:49:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 08:49:11.900576 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08e067f-75ac-445c-910a-81eeba9c67d2 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:49:11.910273 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08e067f-75ac-445c-910a-81eeba9c67d2 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:49:11.915198 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6317a31-a789-42e6-832e-c8a4270da439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:49:11.915830 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6317a31-a789-42e6-832e-c8a4270da439 admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:49:11.916109 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6317a31-a789-42e6-832e-c8a4270da439 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:49:11.916376 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6317a31-a789-42e6-832e-c8a4270da439 admin admin] PATH_INFO: `/v3/projects/cfaf16c0f2b24c1fb2aad4da97b82d4d/users/80e5c0e5ec784780be05f3b72ac940b4/roles/44b64db484f24a46bf0b7ced46b519bf` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:49:11.918552 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:49:11.932054 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6317a31-a789-42e6-832e-c8a4270da439 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfaf16c0f2b24c1fb2aad4da97b82d4d, user_id=80e5c0e5ec784780be05f3b72ac940b4, role_id=44b64db484f24a46bf0b7ced46b519bf)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:49:11.934288 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6317a31-a789-42e6-832e-c8a4270da439 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:49:11.999659 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1210/2431] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 08:49:11 2022] PUT /identity/v3/projects/cfaf16c0f2b24c1fb2aad4da97b82d4d/users/80e5c0e5ec784780be05f3b72ac940b4/roles/44b64db484f24a46bf0b7ced46b519bf => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:49:12.010192 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61008793-c925-4173-a710-7a71587dfa08 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:49:12.010726 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61008793-c925-4173-a710-7a71587dfa08 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:49:12.012400 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61008793-c925-4173-a710-7a71587dfa08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:49:12.038738 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-61008793-c925-4173-a710-7a71587dfa08 None None] MFA Rules not processed for user `80e5c0e5ec784780be05f3b72ac940b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:49:12.039950 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-61008793-c925-4173-a710-7a71587dfa08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:49:12.107295 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1222/2432] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:49:12 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:49:12.119580 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6317a31-a789-42e6-832e-c8a4270da439 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:49:12.121333 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-a6317a31-a789-42e6-832e-c8a4270da439 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:49:12.179200 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6317a31-a789-42e6-832e-c8a4270da439 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:49:12.182896 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c5ad47-f48a-43d8-adf4-392c83cfa51f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ba74e28ced4cbba857f54222dfbe41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:49:12.183424 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c5ad47-f48a-43d8-adf4-392c83cfa51f service cinder] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:49:12.183700 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c5ad47-f48a-43d8-adf4-392c83cfa51f service cinder] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:49:12.183966 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c5ad47-f48a-43d8-adf4-392c83cfa51f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:49:12.192557 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c5ad47-f48a-43d8-adf4-392c83cfa51f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:49:12.194006 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c5ad47-f48a-43d8-adf4-392c83cfa51f service cinder] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:49:12.202638 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1211/2433] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:49:12 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:49:22.104358 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=81,keystone:INSERT=3 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:49:22.198688 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=33,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:49:41.407287 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa8263c-974f-4f2b-88ee-a36609796d05 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:49:41.407721 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa8263c-974f-4f2b-88ee-a36609796d05 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:49:41.407925 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa8263c-974f-4f2b-88ee-a36609796d05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:49:41.437121 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-8aa8263c-974f-4f2b-88ee-a36609796d05 None None] MFA Rules not processed for user `258ec0f975eb47738f4dac356c0578e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:49:41.438330 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-8aa8263c-974f-4f2b-88ee-a36609796d05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:49:41.513020 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1223/2434] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:49:41 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:49:51.509822 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=37 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:50:07.488378 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c5ad47-f48a-43d8-adf4-392c83cfa51f service cinder] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:50:07.594388 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c5ad47-f48a-43d8-adf4-392c83cfa51f service cinder] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:50:07.598894 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b7507a-6756-46f4-b5a9-b0f031e79beb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:50:07.599508 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b7507a-6756-46f4-b5a9-b0f031e79beb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:50:07.599784 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b7507a-6756-46f4-b5a9-b0f031e79beb admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:50:07.600050 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b7507a-6756-46f4-b5a9-b0f031e79beb admin admin] PATH_INFO: `/v3/users/80e5c0e5ec784780be05f3b72ac940b4` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:50:07.601194 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b7507a-6756-46f4-b5a9-b0f031e79beb admin admin] RBAC: Authorizing `identity:delete_user(user_id=80e5c0e5ec784780be05f3b72ac940b4)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:50:07.603093 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b7507a-6756-46f4-b5a9-b0f031e79beb admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:50:07.803275 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-77b7507a-6756-46f4-b5a9-b0f031e79beb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80e5c0e5ec784780be05f3b72ac940b4'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:50:07.815625 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-77b7507a-6756-46f4-b5a9-b0f031e79beb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80e5c0e5ec784780be05f3b72ac940b4'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:50:07.822411 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-77b7507a-6756-46f4-b5a9-b0f031e79beb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80e5c0e5ec784780be05f3b72ac940b4'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:50:07.823109 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-77b7507a-6756-46f4-b5a9-b0f031e79beb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80e5c0e5ec784780be05f3b72ac940b4'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:50:07.823714 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-77b7507a-6756-46f4-b5a9-b0f031e79beb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80e5c0e5ec784780be05f3b72ac940b4'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:50:07.931182 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1212/2435] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 08:50:07 2022] DELETE /identity/v3/users/80e5c0e5ec784780be05f3b72ac940b4 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:50:07.935796 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd58928b-4b30-4e6c-a627-7711d5780953 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:50:07.936311 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd58928b-4b30-4e6c-a627-7711d5780953 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:50:07.936427 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd58928b-4b30-4e6c-a627-7711d5780953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:50:07.979745 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-bd58928b-4b30-4e6c-a627-7711d5780953 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:50:07.980991 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-bd58928b-4b30-4e6c-a627-7711d5780953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:50:08.043879 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1224/2436] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:50:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:50:08.054253 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b7507a-6756-46f4-b5a9-b0f031e79beb admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:50:08.055718 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-77b7507a-6756-46f4-b5a9-b0f031e79beb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:50:08.099369 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:50:08.152497 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b7507a-6756-46f4-b5a9-b0f031e79beb admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:50:08.157316 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1729711-c645-42fc-8693-cbbffe628cb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:50:08.157923 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1729711-c645-42fc-8693-cbbffe628cb4 service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:50:08.158198 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1729711-c645-42fc-8693-cbbffe628cb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:50:08.158553 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1729711-c645-42fc-8693-cbbffe628cb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:50:08.167881 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1729711-c645-42fc-8693-cbbffe628cb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:50:08.169348 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1729711-c645-42fc-8693-cbbffe628cb4 service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:50:08.179113 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1213/2437] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:50:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:50:08.544228 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd58928b-4b30-4e6c-a627-7711d5780953 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:50:08.545673 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-bd58928b-4b30-4e6c-a627-7711d5780953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:50:08.560062 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd58928b-4b30-4e6c-a627-7711d5780953 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:50:08.566531 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b17483-9ca0-49e9-8b79-7f56801b9864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:50:08.568297 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b17483-9ca0-49e9-8b79-7f56801b9864 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:50:08.568297 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b17483-9ca0-49e9-8b79-7f56801b9864 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:50:08.568297 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b17483-9ca0-49e9-8b79-7f56801b9864 admin admin] PATH_INFO: `/v3/projects/cfaf16c0f2b24c1fb2aad4da97b82d4d` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:50:08.574714 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b17483-9ca0-49e9-8b79-7f56801b9864 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cfaf16c0f2b24c1fb2aad4da97b82d4d)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:50:08.576719 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b17483-9ca0-49e9-8b79-7f56801b9864 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:50:08.732079 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-87b17483-9ca0-49e9-8b79-7f56801b9864 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfaf16c0f2b24c1fb2aad4da97b82d4d', 'request_id': 'req-87b17483-9ca0-49e9-8b79-7f56801b9864'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:50:08.738675 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-87b17483-9ca0-49e9-8b79-7f56801b9864 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfaf16c0f2b24c1fb2aad4da97b82d4d', 'request_id': 'req-87b17483-9ca0-49e9-8b79-7f56801b9864'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:50:08.834321 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-87b17483-9ca0-49e9-8b79-7f56801b9864 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfaf16c0f2b24c1fb2aad4da97b82d4d', 'request_id': 'req-87b17483-9ca0-49e9-8b79-7f56801b9864'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:50:08.846674 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-87b17483-9ca0-49e9-8b79-7f56801b9864 admin admin] The token cache is being invalidate because project cfaf16c0f2b24c1fb2aad4da97b82d4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 08:50:08.846964 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-87b17483-9ca0-49e9-8b79-7f56801b9864 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:50:08.847600 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-87b17483-9ca0-49e9-8b79-7f56801b9864 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:50:08.849125 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1225/2438] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 08:50:08 2022] DELETE /identity/v3/projects/cfaf16c0f2b24c1fb2aad4da97b82d4d => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:50:08.859896 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dab2737-5a15-4aa2-9137-9045f59be9d1 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:50:08.860436 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dab2737-5a15-4aa2-9137-9045f59be9d1 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:50:08.860567 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dab2737-5a15-4aa2-9137-9045f59be9d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:50:08.908614 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-3dab2737-5a15-4aa2-9137-9045f59be9d1 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:50:08.909882 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-3dab2737-5a15-4aa2-9137-9045f59be9d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:50:08.968634 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1214/2439] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:50:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 08:50:08.976584 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2f058d-4298-4a0b-826c-4d7597c29222 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:50:08.977114 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2f058d-4298-4a0b-826c-4d7597c29222 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:50:08.977239 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2f058d-4298-4a0b-826c-4d7597c29222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:50:08.997078 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-ae2f058d-4298-4a0b-826c-4d7597c29222 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:50:08.998263 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-ae2f058d-4298-4a0b-826c-4d7597c29222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:50:09.006606 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1226/2440] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:50:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:50:09.010803 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dab2737-5a15-4aa2-9137-9045f59be9d1 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:50:09.021071 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dab2737-5a15-4aa2-9137-9045f59be9d1 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:50:09.025513 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a071a28-0066-40a7-affc-1f7c7a9c1fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:50:09.026098 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a071a28-0066-40a7-affc-1f7c7a9c1fb9 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:50:09.026373 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a071a28-0066-40a7-affc-1f7c7a9c1fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:50:09.026638 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a071a28-0066-40a7-affc-1f7c7a9c1fb9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:50:09.027268 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a071a28-0066-40a7-affc-1f7c7a9c1fb9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 08:50:09.027564 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a071a28-0066-40a7-affc-1f7c7a9c1fb9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:50:09.029184 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a071a28-0066-40a7-affc-1f7c7a9c1fb9 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:50:09.041271 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1215/2441] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Aug 10 08:50:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 08:50:09.045424 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2f058d-4298-4a0b-826c-4d7597c29222 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:50:09.055250 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2f058d-4298-4a0b-826c-4d7597c29222 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:50:09.059866 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7148c80-b484-426f-8649-a94845e52bef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:50:09.060460 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7148c80-b484-426f-8649-a94845e52bef admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:50:09.060735 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7148c80-b484-426f-8649-a94845e52bef admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:50:09.061031 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7148c80-b484-426f-8649-a94845e52bef admin admin] PATH_INFO: `/v3/projects` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:50:09.061639 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7148c80-b484-426f-8649-a94845e52bef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:50:09.063689 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7148c80-b484-426f-8649-a94845e52bef admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:50:09.140110 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1227/2442] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 08:50:09 2022] POST /identity/v3/projects => generated 373 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 08:50:09.145804 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a071a28-0066-40a7-affc-1f7c7a9c1fb9 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:50:09.156759 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a071a28-0066-40a7-affc-1f7c7a9c1fb9 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:50:09.214428 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389b87b3-1122-4b4b-bc38-90c194adc9c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:50:09.215041 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389b87b3-1122-4b4b-bc38-90c194adc9c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:50:09.215313 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389b87b3-1122-4b4b-bc38-90c194adc9c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:50:09.215578 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389b87b3-1122-4b4b-bc38-90c194adc9c0 admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:50:09.216204 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389b87b3-1122-4b4b-bc38-90c194adc9c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:50:09.218166 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389b87b3-1122-4b4b-bc38-90c194adc9c0 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:50:09.298759 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:50:09.306077 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1216/2443] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 08:50:09 2022] POST /identity/v3/users => generated 287 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 08:50:09.312201 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7148c80-b484-426f-8649-a94845e52bef admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:50:09.323183 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7148c80-b484-426f-8649-a94845e52bef admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:50:09.327883 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec95053d-5732-4005-8d1f-f549e9f7d6cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:50:09.328469 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec95053d-5732-4005-8d1f-f549e9f7d6cc admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:50:09.328748 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec95053d-5732-4005-8d1f-f549e9f7d6cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:50:09.329060 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec95053d-5732-4005-8d1f-f549e9f7d6cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:50:09.329698 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec95053d-5732-4005-8d1f-f549e9f7d6cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:50:09.331348 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec95053d-5732-4005-8d1f-f549e9f7d6cc admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:50:09.341499 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1228/2444] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:50:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:50:09.346380 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389b87b3-1122-4b4b-bc38-90c194adc9c0 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:50:09.357469 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389b87b3-1122-4b4b-bc38-90c194adc9c0 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:50:09.362914 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7953ceb-8980-4c5d-9895-7547dbc69d93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:50:09.363901 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7953ceb-8980-4c5d-9895-7547dbc69d93 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:50:09.363901 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7953ceb-8980-4c5d-9895-7547dbc69d93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:50:09.364109 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7953ceb-8980-4c5d-9895-7547dbc69d93 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:50:09.364690 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7953ceb-8980-4c5d-9895-7547dbc69d93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:50:09.366321 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7953ceb-8980-4c5d-9895-7547dbc69d93 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:50:09.376907 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1217/2445] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 08:50:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 08:50:09.381091 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec95053d-5732-4005-8d1f-f549e9f7d6cc admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:50:09.392048 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec95053d-5732-4005-8d1f-f549e9f7d6cc admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:50:09.396655 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d00779-50b3-482e-84c9-e7b5e63c5c10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:50:09.397365 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d00779-50b3-482e-84c9-e7b5e63c5c10 admin admin] REQUEST_METHOD: `PUT` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:50:09.397642 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d00779-50b3-482e-84c9-e7b5e63c5c10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:50:09.397909 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d00779-50b3-482e-84c9-e7b5e63c5c10 admin admin] PATH_INFO: `/v3/projects/7247cedfe93d49a89326bffd1ba47038/users/04a6baa9ef974e99b41fb0e5ed8f6ef3/roles/44b64db484f24a46bf0b7ced46b519bf` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:50:09.413447 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d00779-50b3-482e-84c9-e7b5e63c5c10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7247cedfe93d49a89326bffd1ba47038, user_id=04a6baa9ef974e99b41fb0e5ed8f6ef3, role_id=44b64db484f24a46bf0b7ced46b519bf)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:50:09.416126 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d00779-50b3-482e-84c9-e7b5e63c5c10 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:50:09.479591 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1229/2446] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 08:50:09 2022] PUT /identity/v3/projects/7247cedfe93d49a89326bffd1ba47038/users/04a6baa9ef974e99b41fb0e5ed8f6ef3/roles/44b64db484f24a46bf0b7ced46b519bf => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 08:50:09.490079 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11f5c7c-ea5d-4da6-b2ec-f96ee51be7a6 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:50:09.490499 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11f5c7c-ea5d-4da6-b2ec-f96ee51be7a6 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:50:09.490815 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11f5c7c-ea5d-4da6-b2ec-f96ee51be7a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:50:09.519228 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-f11f5c7c-ea5d-4da6-b2ec-f96ee51be7a6 None None] MFA Rules not processed for user `04a6baa9ef974e99b41fb0e5ed8f6ef3`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:50:09.520446 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-f11f5c7c-ea5d-4da6-b2ec-f96ee51be7a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:50:09.583710 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1218/2447] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:50:09 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:50:09.595647 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d00779-50b3-482e-84c9-e7b5e63c5c10 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:50:09.597335 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-34d00779-50b3-482e-84c9-e7b5e63c5c10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:50:09.601192 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:50:09.677319 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d00779-50b3-482e-84c9-e7b5e63c5c10 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:50:09.681163 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f085b9cb-6da6-49cd-941a-9aa6af37e4ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ba74e28ced4cbba857f54222dfbe41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:50:09.681768 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f085b9cb-6da6-49cd-941a-9aa6af37e4ed service cinder] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:50:09.682047 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f085b9cb-6da6-49cd-941a-9aa6af37e4ed service cinder] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:50:09.682317 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f085b9cb-6da6-49cd-941a-9aa6af37e4ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:50:09.691060 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f085b9cb-6da6-49cd-941a-9aa6af37e4ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:50:09.692478 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f085b9cb-6da6-49cd-941a-9aa6af37e4ed service cinder] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:50:09.701100 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1230/2448] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:50:09 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 08:50:19.580941 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=38 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:50:19.697150 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=34 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:50:20.929941 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11f5c7c-ea5d-4da6-b2ec-f96ee51be7a6 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:50:21.002189 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11f5c7c-ea5d-4da6-b2ec-f96ee51be7a6 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:50:21.011025 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121fd7dc-adaf-4520-8aaf-fdf356a062a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:50:21.011723 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121fd7dc-adaf-4520-8aaf-fdf356a062a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:50:21.012002 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121fd7dc-adaf-4520-8aaf-fdf356a062a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:50:21.012525 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121fd7dc-adaf-4520-8aaf-fdf356a062a4 admin admin] PATH_INFO: `/v3/users/04a6baa9ef974e99b41fb0e5ed8f6ef3` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:50:21.014381 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121fd7dc-adaf-4520-8aaf-fdf356a062a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=04a6baa9ef974e99b41fb0e5ed8f6ef3)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:50:21.016841 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121fd7dc-adaf-4520-8aaf-fdf356a062a4 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:50:21.219035 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-121fd7dc-adaf-4520-8aaf-fdf356a062a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04a6baa9ef974e99b41fb0e5ed8f6ef3'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:50:21.230761 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-121fd7dc-adaf-4520-8aaf-fdf356a062a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04a6baa9ef974e99b41fb0e5ed8f6ef3'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:50:21.237564 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-121fd7dc-adaf-4520-8aaf-fdf356a062a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04a6baa9ef974e99b41fb0e5ed8f6ef3'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:50:21.238235 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-121fd7dc-adaf-4520-8aaf-fdf356a062a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04a6baa9ef974e99b41fb0e5ed8f6ef3'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:50:21.238863 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-121fd7dc-adaf-4520-8aaf-fdf356a062a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04a6baa9ef974e99b41fb0e5ed8f6ef3'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:50:21.320057 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1219/2449] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 08:50:20 2022] DELETE /identity/v3/users/04a6baa9ef974e99b41fb0e5ed8f6ef3 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:50:21.325083 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6100239-6680-41cf-8f33-c8ca7c2df07f None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:50:21.325571 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6100239-6680-41cf-8f33-c8ca7c2df07f None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:50:21.325689 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6100239-6680-41cf-8f33-c8ca7c2df07f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:50:21.345155 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-b6100239-6680-41cf-8f33-c8ca7c2df07f None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:50:21.346347 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-b6100239-6680-41cf-8f33-c8ca7c2df07f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:50:21.404490 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1231/2450] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 08:50:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 08:50:21.414381 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121fd7dc-adaf-4520-8aaf-fdf356a062a4 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:50:21.415769 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-121fd7dc-adaf-4520-8aaf-fdf356a062a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:50:21.473140 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121fd7dc-adaf-4520-8aaf-fdf356a062a4 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:50:21.476964 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820f6003-132f-454a-a5b2-2e4263c9c973 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:50:21.477613 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820f6003-132f-454a-a5b2-2e4263c9c973 service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:50:21.477882 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820f6003-132f-454a-a5b2-2e4263c9c973 service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:50:21.478156 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820f6003-132f-454a-a5b2-2e4263c9c973 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:50:21.487982 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820f6003-132f-454a-a5b2-2e4263c9c973 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:50:21.489472 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820f6003-132f-454a-a5b2-2e4263c9c973 service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:50:21.499158 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1220/2451] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:50:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:50:21.893152 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6100239-6680-41cf-8f33-c8ca7c2df07f None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:50:21.894673 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-b6100239-6680-41cf-8f33-c8ca7c2df07f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:50:21.908300 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6100239-6680-41cf-8f33-c8ca7c2df07f None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:50:21.913301 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40553ab-98e1-4407-8fbb-97dbb7d51c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:50:21.918992 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40553ab-98e1-4407-8fbb-97dbb7d51c55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:50:21.918992 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40553ab-98e1-4407-8fbb-97dbb7d51c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:50:21.918992 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40553ab-98e1-4407-8fbb-97dbb7d51c55 admin admin] PATH_INFO: `/v3/projects/7247cedfe93d49a89326bffd1ba47038` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:50:21.918992 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40553ab-98e1-4407-8fbb-97dbb7d51c55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7247cedfe93d49a89326bffd1ba47038)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:50:21.918992 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40553ab-98e1-4407-8fbb-97dbb7d51c55 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:50:22.022777 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-d40553ab-98e1-4407-8fbb-97dbb7d51c55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7247cedfe93d49a89326bffd1ba47038', 'request_id': 'req-d40553ab-98e1-4407-8fbb-97dbb7d51c55'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:50:22.028073 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-d40553ab-98e1-4407-8fbb-97dbb7d51c55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7247cedfe93d49a89326bffd1ba47038', 'request_id': 'req-d40553ab-98e1-4407-8fbb-97dbb7d51c55'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:50:22.096624 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-d40553ab-98e1-4407-8fbb-97dbb7d51c55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7247cedfe93d49a89326bffd1ba47038', 'request_id': 'req-d40553ab-98e1-4407-8fbb-97dbb7d51c55'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:50:22.107703 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-d40553ab-98e1-4407-8fbb-97dbb7d51c55 admin admin] The token cache is being invalidate because project 7247cedfe93d49a89326bffd1ba47038 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 08:50:22.107988 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-d40553ab-98e1-4407-8fbb-97dbb7d51c55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:50:22.108644 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-d40553ab-98e1-4407-8fbb-97dbb7d51c55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:50:22.111343 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1232/2452] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 08:50:21 2022] DELETE /identity/v3/projects/7247cedfe93d49a89326bffd1ba47038 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:50:25.476324 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c24cbbb-e558-4ac9-b6f3-b5aa61c27998 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:50:25.476829 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c24cbbb-e558-4ac9-b6f3-b5aa61c27998 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:50:25.476962 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c24cbbb-e558-4ac9-b6f3-b5aa61c27998 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:50:25.506409 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-6c24cbbb-e558-4ac9-b6f3-b5aa61c27998 None None] MFA Rules not processed for user `c140d31bd2b84c48a170e3ca90a041ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:50:25.507646 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-6c24cbbb-e558-4ac9-b6f3-b5aa61c27998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:50:25.517051 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:50:25.569852 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1221/2453] 10.222.0.57 () {58 vars in 1009 bytes} [Wed Aug 10 08:50:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:50:25.706955 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40553ab-98e1-4407-8fbb-97dbb7d51c55 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:50:25.708552 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-d40553ab-98e1-4407-8fbb-97dbb7d51c55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:50:25.779506 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40553ab-98e1-4407-8fbb-97dbb7d51c55 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:50:25.785337 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079dc2e7-56ef-47cf-b232-4eb1efb7188b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4b26ba3a9e04c38b52db2a46198a53c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:50:25.785943 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079dc2e7-56ef-47cf-b232-4eb1efb7188b service swift] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:50:25.786223 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079dc2e7-56ef-47cf-b232-4eb1efb7188b service swift] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:50:25.786491 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079dc2e7-56ef-47cf-b232-4eb1efb7188b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:50:25.795634 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079dc2e7-56ef-47cf-b232-4eb1efb7188b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:50:25.798197 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079dc2e7-56ef-47cf-b232-4eb1efb7188b service swift] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:50:25.806235 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1233/2454] 10.222.0.57 () {60 vars in 1384 bytes} [Wed Aug 10 08:50:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 08:50:28.027923 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c24cbbb-e558-4ac9-b6f3-b5aa61c27998 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:50:28.029427 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-6c24cbbb-e558-4ac9-b6f3-b5aa61c27998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:50:28.094445 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c24cbbb-e558-4ac9-b6f3-b5aa61c27998 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:50:28.098254 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a314bc0-3a23-4777-b4f7-6710b62b7b17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:50:28.098869 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a314bc0-3a23-4777-b4f7-6710b62b7b17 service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:50:28.099154 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a314bc0-3a23-4777-b4f7-6710b62b7b17 service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:50:28.099426 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a314bc0-3a23-4777-b4f7-6710b62b7b17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:50:28.101200 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-4a314bc0-3a23-4777-b4f7-6710b62b7b17 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:50:28.184324 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a314bc0-3a23-4777-b4f7-6710b62b7b17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:50:28.187760 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a314bc0-3a23-4777-b4f7-6710b62b7b17 service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:50:28.207393 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1222/2455] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:50:28 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:50:33.704982 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079dc2e7-56ef-47cf-b232-4eb1efb7188b service swift] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:50:33.707242 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-079dc2e7-56ef-47cf-b232-4eb1efb7188b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:50:33.730164 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:50:33.825132 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079dc2e7-56ef-47cf-b232-4eb1efb7188b service swift] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:50:33.835157 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846d51d2-5931-4757-8365-6ec0a35e1c63 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2912307c56594125b157f1cfdf7c96a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:50:33.835157 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846d51d2-5931-4757-8365-6ec0a35e1c63 service placement] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:50:33.835157 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846d51d2-5931-4757-8365-6ec0a35e1c63 service placement] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:50:33.835157 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846d51d2-5931-4757-8365-6ec0a35e1c63 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:50:33.836154 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-846d51d2-5931-4757-8365-6ec0a35e1c63 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:50:33.848801 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846d51d2-5931-4757-8365-6ec0a35e1c63 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:50:33.850447 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846d51d2-5931-4757-8365-6ec0a35e1c63 service placement] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:50:33.863883 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1234/2456] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:50:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:50:36.254043 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a314bc0-3a23-4777-b4f7-6710b62b7b17 service neutron] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:50:36.264322 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a314bc0-3a23-4777-b4f7-6710b62b7b17 service neutron] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:50:36.271724 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dfb8b4-2bac-428d-b32f-7bfc823f7de0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:50:36.274685 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dfb8b4-2bac-428d-b32f-7bfc823f7de0 service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:50:36.274685 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dfb8b4-2bac-428d-b32f-7bfc823f7de0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:50:36.274685 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dfb8b4-2bac-428d-b32f-7bfc823f7de0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:50:36.274685 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-65dfb8b4-2bac-428d-b32f-7bfc823f7de0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:50:36.366966 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dfb8b4-2bac-428d-b32f-7bfc823f7de0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:50:36.368676 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dfb8b4-2bac-428d-b32f-7bfc823f7de0 service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:50:36.377535 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:50:36.397340 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1223/2457] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:50:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:50:40.750786 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846d51d2-5931-4757-8365-6ec0a35e1c63 service placement] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:50:40.752291 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-846d51d2-5931-4757-8365-6ec0a35e1c63 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:50:40.765601 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846d51d2-5931-4757-8365-6ec0a35e1c63 service placement] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:50:40.770759 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619cae98-a148-4bc3-a1a6-cd14d475061c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2912307c56594125b157f1cfdf7c96a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:50:40.774414 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619cae98-a148-4bc3-a1a6-cd14d475061c service placement] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:50:40.774414 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619cae98-a148-4bc3-a1a6-cd14d475061c service placement] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:50:40.774414 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619cae98-a148-4bc3-a1a6-cd14d475061c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:50:40.774414 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-619cae98-a148-4bc3-a1a6-cd14d475061c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:50:40.786137 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619cae98-a148-4bc3-a1a6-cd14d475061c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:50:40.787975 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619cae98-a148-4bc3-a1a6-cd14d475061c service placement] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:50:40.801408 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1235/2458] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:50:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:50:43.206961 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dfb8b4-2bac-428d-b32f-7bfc823f7de0 service neutron] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:50:43.208451 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-65dfb8b4-2bac-428d-b32f-7bfc823f7de0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:50:43.282680 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dfb8b4-2bac-428d-b32f-7bfc823f7de0 service neutron] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:50:43.291660 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fdef00-1c54-44ab-8ab1-fe21c6fecb40 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ba74e28ced4cbba857f54222dfbe41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:50:43.291660 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fdef00-1c54-44ab-8ab1-fe21c6fecb40 service cinder] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:50:43.291660 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fdef00-1c54-44ab-8ab1-fe21c6fecb40 service cinder] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:50:43.291660 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fdef00-1c54-44ab-8ab1-fe21c6fecb40 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:50:43.293047 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-31fdef00-1c54-44ab-8ab1-fe21c6fecb40 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:50:43.367678 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fdef00-1c54-44ab-8ab1-fe21c6fecb40 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:50:43.369147 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fdef00-1c54-44ab-8ab1-fe21c6fecb40 service cinder] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:50:43.378854 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1224/2459] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:50:43 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:50:44.421935 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5b9e1d-8f63-4a82-9eb2-6027833df665 None None] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:50:44.427048 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5b9e1d-8f63-4a82-9eb2-6027833df665 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:50:44.427048 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5b9e1d-8f63-4a82-9eb2-6027833df665 None None] PATH_INFO: `/` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:50:44.427048 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1236/2460] 10.222.0.57 () {54 vars in 921 bytes} [Wed Aug 10 08:50:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:50:44.431202 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65b1c23-da3e-4111-b48f-1cc60fd84843 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:50:44.431693 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65b1c23-da3e-4111-b48f-1cc60fd84843 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:50:44.431790 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65b1c23-da3e-4111-b48f-1cc60fd84843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:50:44.471046 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-b65b1c23-da3e-4111-b48f-1cc60fd84843 None None] MFA Rules not processed for user `a943008193914f76a663d489bd73e873`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:50:44.472413 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-b65b1c23-da3e-4111-b48f-1cc60fd84843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:50:44.538487 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5b9e1d-8f63-4a82-9eb2-6027833df665 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:50:44.540225 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-0a5b9e1d-8f63-4a82-9eb2-6027833df665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:50:44.557561 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1225/2461] 10.222.0.57 () {58 vars in 1018 bytes} [Wed Aug 10 08:50:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:50:44.559226 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5b9e1d-8f63-4a82-9eb2-6027833df665 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:50:44.565675 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2acb37a-7a67-4e73-a6a3-ca195e10018e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:50:44.566302 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2acb37a-7a67-4e73-a6a3-ca195e10018e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:50:44.566580 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2acb37a-7a67-4e73-a6a3-ca195e10018e admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:50:44.566857 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2acb37a-7a67-4e73-a6a3-ca195e10018e admin admin] PATH_INFO: `/v3/users/1d346511f25c4f059f0dedfb3467ed06` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:50:44.567984 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2acb37a-7a67-4e73-a6a3-ca195e10018e admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d346511f25c4f059f0dedfb3467ed06)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:50:44.570007 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65b1c23-da3e-4111-b48f-1cc60fd84843 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:50:44.570664 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2acb37a-7a67-4e73-a6a3-ca195e10018e admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:50:44.571626 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-b65b1c23-da3e-4111-b48f-1cc60fd84843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:50:44.605469 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:50:44.631812 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65b1c23-da3e-4111-b48f-1cc60fd84843 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:50:44.635478 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c136c1fb-d8d1-48ba-909f-bf5e1f27952e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3263af99fc2f48e79fd1e5b8a1e555a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:50:44.636102 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c136c1fb-d8d1-48ba-909f-bf5e1f27952e service glance] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:50:44.636386 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c136c1fb-d8d1-48ba-909f-bf5e1f27952e service glance] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:50:44.636659 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c136c1fb-d8d1-48ba-909f-bf5e1f27952e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:50:44.647709 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c136c1fb-d8d1-48ba-909f-bf5e1f27952e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:50:44.649195 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c136c1fb-d8d1-48ba-909f-bf5e1f27952e service glance] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:50:44.660198 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1226/2462] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:50:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 08:50:44.735305 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-a2acb37a-7a67-4e73-a6a3-ca195e10018e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d346511f25c4f059f0dedfb3467ed06'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:50:44.747447 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-a2acb37a-7a67-4e73-a6a3-ca195e10018e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d346511f25c4f059f0dedfb3467ed06'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:50:44.755178 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-a2acb37a-7a67-4e73-a6a3-ca195e10018e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d346511f25c4f059f0dedfb3467ed06'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:50:44.851098 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-a2acb37a-7a67-4e73-a6a3-ca195e10018e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d346511f25c4f059f0dedfb3467ed06'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:50:44.851739 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-a2acb37a-7a67-4e73-a6a3-ca195e10018e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d346511f25c4f059f0dedfb3467ed06'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:50:44.858291 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1237/2463] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 08:50:44 2022] DELETE /identity/v3/users/1d346511f25c4f059f0dedfb3467ed06 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 08:50:44.991612 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395e1220-3acc-4861-b02a-94347dd88ca4 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:50:44.991895 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395e1220-3acc-4861-b02a-94347dd88ca4 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:50:44.992166 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395e1220-3acc-4861-b02a-94347dd88ca4 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:50:44.993768 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1227/2464] 10.222.0.57 () {54 vars in 921 bytes} [Wed Aug 10 08:50:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 08:50:45.000437 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b0f4b5-765b-4c74-b586-afdf328f8b8e None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:50:45.000716 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b0f4b5-765b-4c74-b586-afdf328f8b8e None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:50:45.001075 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b0f4b5-765b-4c74-b586-afdf328f8b8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:50:45.023319 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-35b0f4b5-765b-4c74-b586-afdf328f8b8e None None] MFA Rules not processed for user `a943008193914f76a663d489bd73e873`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:50:45.024536 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-35b0f4b5-765b-4c74-b586-afdf328f8b8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:50:45.067672 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:50:45.095746 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1238/2465] 10.222.0.57 () {58 vars in 1018 bytes} [Wed Aug 10 08:50:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:50:45.106430 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395e1220-3acc-4861-b02a-94347dd88ca4 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:50:45.108556 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-395e1220-3acc-4861-b02a-94347dd88ca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:50:45.119458 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b0f4b5-765b-4c74-b586-afdf328f8b8e None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:50:45.121838 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-35b0f4b5-765b-4c74-b586-afdf328f8b8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:50:45.183921 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395e1220-3acc-4861-b02a-94347dd88ca4 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:50:45.187788 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe3b002-4a5b-4d66-a5f2-8b6fcfdf5c37 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:50:45.188389 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe3b002-4a5b-4d66-a5f2-8b6fcfdf5c37 service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:50:45.188665 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe3b002-4a5b-4d66-a5f2-8b6fcfdf5c37 service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:50:45.189087 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe3b002-4a5b-4d66-a5f2-8b6fcfdf5c37 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:50:45.199083 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe3b002-4a5b-4d66-a5f2-8b6fcfdf5c37 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:50:45.200275 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b0f4b5-765b-4c74-b586-afdf328f8b8e None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:50:45.200506 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe3b002-4a5b-4d66-a5f2-8b6fcfdf5c37 service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:50:45.205165 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5799c1b5-16d6-4ec4-b06d-198486df054e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:50:45.205918 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5799c1b5-16d6-4ec4-b06d-198486df054e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:50:45.206204 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5799c1b5-16d6-4ec4-b06d-198486df054e admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:50:45.206480 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5799c1b5-16d6-4ec4-b06d-198486df054e admin admin] PATH_INFO: `/v3/projects/370aff3310134a3fb2823908129ae45a` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:50:45.207617 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5799c1b5-16d6-4ec4-b06d-198486df054e admin admin] RBAC: Authorizing `identity:delete_project(project_id=370aff3310134a3fb2823908129ae45a)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:50:45.209587 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5799c1b5-16d6-4ec4-b06d-198486df054e admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:50:45.212147 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1228/2466] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:50:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:50:45.315398 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-5799c1b5-16d6-4ec4-b06d-198486df054e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '370aff3310134a3fb2823908129ae45a', 'request_id': 'req-5799c1b5-16d6-4ec4-b06d-198486df054e'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:50:45.321567 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-5799c1b5-16d6-4ec4-b06d-198486df054e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '370aff3310134a3fb2823908129ae45a', 'request_id': 'req-5799c1b5-16d6-4ec4-b06d-198486df054e'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:50:45.388995 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-5799c1b5-16d6-4ec4-b06d-198486df054e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '370aff3310134a3fb2823908129ae45a', 'request_id': 'req-5799c1b5-16d6-4ec4-b06d-198486df054e'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:50:45.402818 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-5799c1b5-16d6-4ec4-b06d-198486df054e admin admin] The token cache is being invalidate because project 370aff3310134a3fb2823908129ae45a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 08:50:45.403661 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-5799c1b5-16d6-4ec4-b06d-198486df054e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:50:45.403783 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-5799c1b5-16d6-4ec4-b06d-198486df054e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:50:45.405351 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1239/2467] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 08:50:45 2022] DELETE /identity/v3/projects/370aff3310134a3fb2823908129ae45a => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:50:45.409907 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe3b002-4a5b-4d66-a5f2-8b6fcfdf5c37 service neutron] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:50:45.420678 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe3b002-4a5b-4d66-a5f2-8b6fcfdf5c37 service neutron] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:50:45.478590 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a98354-f00e-4e9b-be09-14ad18fdbea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:50:45.479248 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a98354-f00e-4e9b-be09-14ad18fdbea0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:50:45.479433 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a98354-f00e-4e9b-be09-14ad18fdbea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:50:45.479689 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a98354-f00e-4e9b-be09-14ad18fdbea0 admin admin] PATH_INFO: `/v3/users/f553fa4ae3ca41d68d0872e13d8161cd` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:50:45.481038 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a98354-f00e-4e9b-be09-14ad18fdbea0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f553fa4ae3ca41d68d0872e13d8161cd)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:50:45.482835 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a98354-f00e-4e9b-be09-14ad18fdbea0 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:50:45.661125 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-52a98354-f00e-4e9b-be09-14ad18fdbea0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f553fa4ae3ca41d68d0872e13d8161cd'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:50:45.674271 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-52a98354-f00e-4e9b-be09-14ad18fdbea0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f553fa4ae3ca41d68d0872e13d8161cd'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:50:45.681813 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-52a98354-f00e-4e9b-be09-14ad18fdbea0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f553fa4ae3ca41d68d0872e13d8161cd'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:50:45.682527 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-52a98354-f00e-4e9b-be09-14ad18fdbea0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f553fa4ae3ca41d68d0872e13d8161cd'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:50:45.683127 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-52a98354-f00e-4e9b-be09-14ad18fdbea0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f553fa4ae3ca41d68d0872e13d8161cd'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:50:45.685030 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:50:45.747802 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1229/2468] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 08:50:45 2022] DELETE /identity/v3/users/f553fa4ae3ca41d68d0872e13d8161cd => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:50:45.966894 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5799c1b5-16d6-4ec4-b06d-198486df054e admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:50:45.968602 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-5799c1b5-16d6-4ec4-b06d-198486df054e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:50:45.982341 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5799c1b5-16d6-4ec4-b06d-198486df054e admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:50:46.039132 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cd83de-3698-4734-9756-851185622cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:50:46.039691 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cd83de-3698-4734-9756-851185622cbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:50:46.039968 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cd83de-3698-4734-9756-851185622cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:50:46.040238 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cd83de-3698-4734-9756-851185622cbe admin admin] PATH_INFO: `/v3/projects/b2ca359757dd4369911e58622104c26d` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:50:46.041382 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cd83de-3698-4734-9756-851185622cbe admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2ca359757dd4369911e58622104c26d)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:50:46.043297 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cd83de-3698-4734-9756-851185622cbe admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:50:46.049603 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:50:46.174895 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-25cd83de-3698-4734-9756-851185622cbe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2ca359757dd4369911e58622104c26d', 'request_id': 'req-25cd83de-3698-4734-9756-851185622cbe'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:50:46.180024 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-25cd83de-3698-4734-9756-851185622cbe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2ca359757dd4369911e58622104c26d', 'request_id': 'req-25cd83de-3698-4734-9756-851185622cbe'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:50:46.256418 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-25cd83de-3698-4734-9756-851185622cbe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2ca359757dd4369911e58622104c26d', 'request_id': 'req-25cd83de-3698-4734-9756-851185622cbe'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:50:46.268176 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-25cd83de-3698-4734-9756-851185622cbe admin admin] The token cache is being invalidate because project b2ca359757dd4369911e58622104c26d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 08:50:46.268465 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-25cd83de-3698-4734-9756-851185622cbe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:50:46.269133 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-25cd83de-3698-4734-9756-851185622cbe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 08:50:46.270621 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1240/2469] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 08:50:45 2022] DELETE /identity/v3/projects/b2ca359757dd4369911e58622104c26d => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 08:50:55.759494 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=2 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:50:56.264846 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=22,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:51:27.841462 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a98354-f00e-4e9b-be09-14ad18fdbea0 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:51:27.842881 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-52a98354-f00e-4e9b-be09-14ad18fdbea0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:51:27.900809 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a98354-f00e-4e9b-be09-14ad18fdbea0 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:51:27.949150 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c9a324-0f98-4387-b976-b5f343485977 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:51:27.949688 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c9a324-0f98-4387-b976-b5f343485977 service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:51:27.949947 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c9a324-0f98-4387-b976-b5f343485977 service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:51:27.950212 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c9a324-0f98-4387-b976-b5f343485977 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:51:27.951927 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-d0c9a324-0f98-4387-b976-b5f343485977 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:51:28.023162 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c9a324-0f98-4387-b976-b5f343485977 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:51:28.024699 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c9a324-0f98-4387-b976-b5f343485977 service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:51:28.036548 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1230/2470] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:51:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:51:31.823783 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cd83de-3698-4734-9756-851185622cbe admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:51:31.825307 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-25cd83de-3698-4734-9756-851185622cbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:51:31.840302 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cd83de-3698-4734-9756-851185622cbe admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:51:31.844014 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d80341b-d7a9-4b90-b43b-f6dae73ff70d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:51:31.844788 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d80341b-d7a9-4b90-b43b-f6dae73ff70d service neutron] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:51:31.845090 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d80341b-d7a9-4b90-b43b-f6dae73ff70d service neutron] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:51:31.845366 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d80341b-d7a9-4b90-b43b-f6dae73ff70d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:51:31.847060 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-0d80341b-d7a9-4b90-b43b-f6dae73ff70d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:51:31.860856 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d80341b-d7a9-4b90-b43b-f6dae73ff70d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:51:31.862314 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d80341b-d7a9-4b90-b43b-f6dae73ff70d service neutron] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:51:31.873101 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1241/2471] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:51:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:51:35.626571 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825495c0-52c7-49e5-9221-bcbd5fefc049 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:51:35.627041 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825495c0-52c7-49e5-9221-bcbd5fefc049 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:51:35.627254 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825495c0-52c7-49e5-9221-bcbd5fefc049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:51:35.647411 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-825495c0-52c7-49e5-9221-bcbd5fefc049 None None] MFA Rules not processed for user `c140d31bd2b84c48a170e3ca90a041ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:51:35.648610 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-825495c0-52c7-49e5-9221-bcbd5fefc049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:51:35.702275 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1231/2472] 10.222.0.57 () {58 vars in 1009 bytes} [Wed Aug 10 08:51:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:51:35.821472 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d80341b-d7a9-4b90-b43b-f6dae73ff70d service neutron] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:51:35.822920 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-0d80341b-d7a9-4b90-b43b-f6dae73ff70d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:51:35.889615 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d80341b-d7a9-4b90-b43b-f6dae73ff70d service neutron] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:51:35.895121 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60370ff4-c910-4020-8a2b-b6d639bcd4bf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4b26ba3a9e04c38b52db2a46198a53c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:51:35.895717 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60370ff4-c910-4020-8a2b-b6d639bcd4bf service swift] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:51:35.895989 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60370ff4-c910-4020-8a2b-b6d639bcd4bf service swift] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:51:35.896254 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60370ff4-c910-4020-8a2b-b6d639bcd4bf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:51:35.903952 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60370ff4-c910-4020-8a2b-b6d639bcd4bf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:51:35.905559 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60370ff4-c910-4020-8a2b-b6d639bcd4bf service swift] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:51:35.913555 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1242/2473] 10.222.0.57 () {60 vars in 1384 bytes} [Wed Aug 10 08:51:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 08:51:45.695475 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=99 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:51:45.910528 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=38 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:51:57.946859 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9195e64-9ac1-41ff-8a1e-0720e51278ab None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:51:57.947351 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9195e64-9ac1-41ff-8a1e-0720e51278ab None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:51:57.947448 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9195e64-9ac1-41ff-8a1e-0720e51278ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:51:57.996723 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-c9195e64-9ac1-41ff-8a1e-0720e51278ab None None] MFA Rules not processed for user `c140d31bd2b84c48a170e3ca90a041ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 08:51:57.998019 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-c9195e64-9ac1-41ff-8a1e-0720e51278ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:51:58.015103 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1232/2474] 10.222.0.57 () {58 vars in 967 bytes} [Wed Aug 10 08:51:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 08:51:58.028101 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60370ff4-c910-4020-8a2b-b6d639bcd4bf service swift] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:51:58.049468 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60370ff4-c910-4020-8a2b-b6d639bcd4bf service swift] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:51:58.055171 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b249fddc-754a-4d69-867e-eff5dfa18e49 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4b26ba3a9e04c38b52db2a46198a53c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:51:58.055767 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b249fddc-754a-4d69-867e-eff5dfa18e49 service swift] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:51:58.056047 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b249fddc-754a-4d69-867e-eff5dfa18e49 service swift] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:51:58.056319 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b249fddc-754a-4d69-867e-eff5dfa18e49 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:51:58.064551 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b249fddc-754a-4d69-867e-eff5dfa18e49 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:51:58.066308 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b249fddc-754a-4d69-867e-eff5dfa18e49 service swift] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:51:58.074754 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1243/2475] 10.222.0.57 () {60 vars in 1384 bytes} [Wed Aug 10 08:51:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 08:52:03.620514 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9195e64-9ac1-41ff-8a1e-0720e51278ab None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:52:03.621910 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-c9195e64-9ac1-41ff-8a1e-0720e51278ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:52:03.676962 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9195e64-9ac1-41ff-8a1e-0720e51278ab None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:52:03.680464 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a06604-fb4b-484c-887d-0792e07d5fa3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ba74e28ced4cbba857f54222dfbe41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:52:03.681088 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a06604-fb4b-484c-887d-0792e07d5fa3 service cinder] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:52:03.681369 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a06604-fb4b-484c-887d-0792e07d5fa3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:52:03.681637 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a06604-fb4b-484c-887d-0792e07d5fa3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:52:03.683361 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-81a06604-fb4b-484c-887d-0792e07d5fa3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:52:03.760770 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a06604-fb4b-484c-887d-0792e07d5fa3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:52:03.762214 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a06604-fb4b-484c-887d-0792e07d5fa3 service cinder] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:52:03.780818 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1233/2476] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:52:03 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:52:08.071144 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=10 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:52:13.778288 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=79 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:52:16.189445 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b249fddc-754a-4d69-867e-eff5dfa18e49 service swift] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:52:16.191015 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-b249fddc-754a-4d69-867e-eff5dfa18e49 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:52:16.202879 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b249fddc-754a-4d69-867e-eff5dfa18e49 service swift] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:52:16.206958 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103ad1f4-2e38-4801-904d-ffeed9b2ff72 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ba74e28ced4cbba857f54222dfbe41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:52:16.207544 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103ad1f4-2e38-4801-904d-ffeed9b2ff72 service cinder] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:52:16.207822 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103ad1f4-2e38-4801-904d-ffeed9b2ff72 service cinder] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:52:16.208093 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103ad1f4-2e38-4801-904d-ffeed9b2ff72 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:52:16.209784 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-103ad1f4-2e38-4801-904d-ffeed9b2ff72 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:52:16.288948 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103ad1f4-2e38-4801-904d-ffeed9b2ff72 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:52:16.290386 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103ad1f4-2e38-4801-904d-ffeed9b2ff72 service cinder] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:52:16.308590 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1244/2477] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:52:16 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:52:26.305937 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=41 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:52:53.318277 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a06604-fb4b-484c-887d-0792e07d5fa3 service cinder] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:52:53.327910 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a06604-fb4b-484c-887d-0792e07d5fa3 service cinder] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:52:53.331547 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4bce08-4141-4d84-9462-ee5479b8c434 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:52:53.332149 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4bce08-4141-4d84-9462-ee5479b8c434 service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:52:53.332460 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4bce08-4141-4d84-9462-ee5479b8c434 service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:52:53.332740 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4bce08-4141-4d84-9462-ee5479b8c434 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:52:53.335003 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-bf4bce08-4141-4d84-9462-ee5479b8c434 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:52:53.344374 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4bce08-4141-4d84-9462-ee5479b8c434 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:52:53.345815 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4bce08-4141-4d84-9462-ee5479b8c434 service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:52:53.354254 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1234/2478] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:52:53 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:52:55.848809 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103ad1f4-2e38-4801-904d-ffeed9b2ff72 service cinder] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:52:55.857762 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103ad1f4-2e38-4801-904d-ffeed9b2ff72 service cinder] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:52:55.861440 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33602f4e-6277-4eec-81d8-76515af42a89 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ba74e28ced4cbba857f54222dfbe41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:52:55.862038 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33602f4e-6277-4eec-81d8-76515af42a89 service cinder] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:52:55.862319 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33602f4e-6277-4eec-81d8-76515af42a89 service cinder] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:52:55.862591 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33602f4e-6277-4eec-81d8-76515af42a89 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:52:55.864303 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-33602f4e-6277-4eec-81d8-76515af42a89 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:52:55.943585 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33602f4e-6277-4eec-81d8-76515af42a89 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:52:55.945058 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33602f4e-6277-4eec-81d8-76515af42a89 service cinder] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:52:55.963459 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1245/2479] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:52:55 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:52:59.095926 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4bce08-4141-4d84-9462-ee5479b8c434 service neutron] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:52:59.097445 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-bf4bce08-4141-4d84-9462-ee5479b8c434 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:52:59.114170 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4bce08-4141-4d84-9462-ee5479b8c434 service neutron] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:52:59.121095 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21903b4c-f6c1-404b-99b0-ccb27bf22375 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a943008193914f76a663d489bd73e873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:52:59.121851 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21903b4c-f6c1-404b-99b0-ccb27bf22375 service nova] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:52:59.122202 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21903b4c-f6c1-404b-99b0-ccb27bf22375 service nova] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:52:59.122672 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21903b4c-f6c1-404b-99b0-ccb27bf22375 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:52:59.125602 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-21903b4c-f6c1-404b-99b0-ccb27bf22375 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:52:59.139667 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21903b4c-f6c1-404b-99b0-ccb27bf22375 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:52:59.141714 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21903b4c-f6c1-404b-99b0-ccb27bf22375 service nova] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:52:59.154978 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1235/2480] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:52:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:53:05.960976 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=41 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:53:09.150429 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=12 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:55:44.370336 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33602f4e-6277-4eec-81d8-76515af42a89 service cinder] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:55:44.379074 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33602f4e-6277-4eec-81d8-76515af42a89 service cinder] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:55:44.383325 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4539712f-ca29-41ef-843a-41dd4f8030a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ba74e28ced4cbba857f54222dfbe41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:55:44.383930 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4539712f-ca29-41ef-843a-41dd4f8030a1 service cinder] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:55:44.385060 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4539712f-ca29-41ef-843a-41dd4f8030a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:55:44.385347 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4539712f-ca29-41ef-843a-41dd4f8030a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:55:44.387317 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-4539712f-ca29-41ef-843a-41dd4f8030a1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:55:44.468205 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4539712f-ca29-41ef-843a-41dd4f8030a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:55:44.469656 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4539712f-ca29-41ef-843a-41dd4f8030a1 service cinder] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:55:44.487514 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1246/2481] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:55:44 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:55:46.429520 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21903b4c-f6c1-404b-99b0-ccb27bf22375 service nova] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:55:46.430956 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-21903b4c-f6c1-404b-99b0-ccb27bf22375 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:55:46.498463 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21903b4c-f6c1-404b-99b0-ccb27bf22375 service nova] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:55:46.503373 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe64f6f-30e2-47e6-83b2-777527fcd533 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2912307c56594125b157f1cfdf7c96a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:55:46.503974 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe64f6f-30e2-47e6-83b2-777527fcd533 service placement] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:55:46.504252 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe64f6f-30e2-47e6-83b2-777527fcd533 service placement] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:55:46.504524 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe64f6f-30e2-47e6-83b2-777527fcd533 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:55:46.506272 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-2fe64f6f-30e2-47e6-83b2-777527fcd533 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 08:55:46.518291 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe64f6f-30e2-47e6-83b2-777527fcd533 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:55:46.519854 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe64f6f-30e2-47e6-83b2-777527fcd533 service placement] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:55:46.530422 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1236/2482] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:55:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:55:54.484818 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=41 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:55:56.526279 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=32 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:57:03.582809 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4539712f-ca29-41ef-843a-41dd4f8030a1 service cinder] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:57:03.592633 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4539712f-ca29-41ef-843a-41dd4f8030a1 service cinder] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:57:03.596431 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f81c0fb-d563-4820-a1d3-12722ecd3814 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ba74e28ced4cbba857f54222dfbe41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:57:03.600715 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f81c0fb-d563-4820-a1d3-12722ecd3814 service cinder] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:57:03.600715 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f81c0fb-d563-4820-a1d3-12722ecd3814 service cinder] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:57:03.600715 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f81c0fb-d563-4820-a1d3-12722ecd3814 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:57:03.606311 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f81c0fb-d563-4820-a1d3-12722ecd3814 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:57:03.607873 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f81c0fb-d563-4820-a1d3-12722ecd3814 service cinder] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:57:03.616733 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1247/2483] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:57:03 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:57:13.612660 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=6 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:57:15.064297 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe64f6f-30e2-47e6-83b2-777527fcd533 service placement] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:57:15.074030 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe64f6f-30e2-47e6-83b2-777527fcd533 service placement] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:57:15.077787 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6909443-5955-4607-b18e-906fba98380a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ba74e28ced4cbba857f54222dfbe41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:57:15.078387 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6909443-5955-4607-b18e-906fba98380a service cinder] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:57:15.078665 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6909443-5955-4607-b18e-906fba98380a service cinder] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:57:15.079025 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6909443-5955-4607-b18e-906fba98380a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:57:15.087626 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6909443-5955-4607-b18e-906fba98380a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:57:15.089692 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6909443-5955-4607-b18e-906fba98380a service cinder] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:57:15.098492 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1237/2484] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:57:15 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:57:16.617603 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f81c0fb-d563-4820-a1d3-12722ecd3814 service cinder] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:57:16.627623 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f81c0fb-d563-4820-a1d3-12722ecd3814 service cinder] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:57:16.631265 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc73541-b8f1-4114-abc7-6eedae72009b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:57:16.631882 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc73541-b8f1-4114-abc7-6eedae72009b service neutron] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:57:16.632161 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc73541-b8f1-4114-abc7-6eedae72009b service neutron] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:57:16.632429 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc73541-b8f1-4114-abc7-6eedae72009b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:57:16.643268 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc73541-b8f1-4114-abc7-6eedae72009b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:57:16.644689 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc73541-b8f1-4114-abc7-6eedae72009b service neutron] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:57:16.655717 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1248/2485] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:57:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:57:17.848614 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6909443-5955-4607-b18e-906fba98380a service cinder] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:57:17.858382 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6909443-5955-4607-b18e-906fba98380a service cinder] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:57:17.862074 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6829a61-335b-445a-a277-c810a007690f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:57:17.862663 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6829a61-335b-445a-a277-c810a007690f service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:57:17.862940 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6829a61-335b-445a-a277-c810a007690f service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:57:17.863203 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6829a61-335b-445a-a277-c810a007690f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:57:17.874244 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6829a61-335b-445a-a277-c810a007690f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:57:17.875671 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6829a61-335b-445a-a277-c810a007690f service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:57:17.887674 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1238/2486] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:57:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:57:26.650783 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=6 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:57:27.881751 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=12 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:57:56.127916 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc73541-b8f1-4114-abc7-6eedae72009b service neutron] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:57:56.138221 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc73541-b8f1-4114-abc7-6eedae72009b service neutron] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:57:56.141813 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93dc430-87c5-44d7-9da5-2cfbd2e6f2bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ba74e28ced4cbba857f54222dfbe41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:57:56.142397 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93dc430-87c5-44d7-9da5-2cfbd2e6f2bc service cinder] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:57:56.142702 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93dc430-87c5-44d7-9da5-2cfbd2e6f2bc service cinder] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:57:56.143236 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93dc430-87c5-44d7-9da5-2cfbd2e6f2bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:57:56.151451 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93dc430-87c5-44d7-9da5-2cfbd2e6f2bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:57:56.154040 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93dc430-87c5-44d7-9da5-2cfbd2e6f2bc service cinder] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:57:56.162818 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1249/2487] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:57:56 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:58:06.158681 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=6 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 08:58:53.852291 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6829a61-335b-445a-a277-c810a007690f service neutron] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 08:58:53.861184 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6829a61-335b-445a-a277-c810a007690f service neutron] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 08:58:53.864716 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ffc98e-97af-4b22-ac93-3c7f73894650 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 08:58:53.865343 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ffc98e-97af-4b22-ac93-3c7f73894650 service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 08:58:53.865620 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ffc98e-97af-4b22-ac93-3c7f73894650 service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 08:58:53.865888 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ffc98e-97af-4b22-ac93-3c7f73894650 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 08:58:53.873329 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ffc98e-97af-4b22-ac93-3c7f73894650 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 08:58:53.874742 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ffc98e-97af-4b22-ac93-3c7f73894650 service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 08:58:53.882543 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1239/2488] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 08:58:53 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 31 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 08:59:03.879031 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=6 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:00:44.237659 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93dc430-87c5-44d7-9da5-2cfbd2e6f2bc service cinder] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:00:44.277429 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93dc430-87c5-44d7-9da5-2cfbd2e6f2bc service cinder] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:00:44.281119 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835b762d-9943-4e6d-abdb-98625f0831a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ba74e28ced4cbba857f54222dfbe41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:00:44.281716 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835b762d-9943-4e6d-abdb-98625f0831a1 service cinder] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:00:44.281990 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835b762d-9943-4e6d-abdb-98625f0831a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:00:44.282255 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835b762d-9943-4e6d-abdb-98625f0831a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:00:44.304321 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835b762d-9943-4e6d-abdb-98625f0831a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:00:44.306786 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835b762d-9943-4e6d-abdb-98625f0831a1 service cinder] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:00:44.315624 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1250/2489] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:00:44 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:00:50.356351 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ffc98e-97af-4b22-ac93-3c7f73894650 service neutron] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:00:50.395802 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ffc98e-97af-4b22-ac93-3c7f73894650 service neutron] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:00:50.401169 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be72da9a-7085-443f-9aad-625394e05d2b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2912307c56594125b157f1cfdf7c96a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:00:50.401775 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be72da9a-7085-443f-9aad-625394e05d2b service placement] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:00:50.402051 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be72da9a-7085-443f-9aad-625394e05d2b service placement] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:00:50.402317 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be72da9a-7085-443f-9aad-625394e05d2b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:00:50.411099 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be72da9a-7085-443f-9aad-625394e05d2b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:00:50.412662 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be72da9a-7085-443f-9aad-625394e05d2b service placement] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:00:50.422675 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1240/2490] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:00:50 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:00:54.311561 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=24 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:01:00.418582 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=18 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:02:03.249561 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835b762d-9943-4e6d-abdb-98625f0831a1 service cinder] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:02:03.334527 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835b762d-9943-4e6d-abdb-98625f0831a1 service cinder] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:02:03.340708 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dd54b8-a191-4304-b283-4dc3de07f7a2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a943008193914f76a663d489bd73e873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:02:03.341347 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dd54b8-a191-4304-b283-4dc3de07f7a2 service nova] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:02:03.341622 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dd54b8-a191-4304-b283-4dc3de07f7a2 service nova] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:02:03.341890 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dd54b8-a191-4304-b283-4dc3de07f7a2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:02:03.349767 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dd54b8-a191-4304-b283-4dc3de07f7a2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:02:03.351297 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dd54b8-a191-4304-b283-4dc3de07f7a2 service nova] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:02:03.360358 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1251/2491] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:02:03 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:02:13.356030 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=41 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:02:15.717522 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be72da9a-7085-443f-9aad-625394e05d2b service placement] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:02:15.719058 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-be72da9a-7085-443f-9aad-625394e05d2b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:02:15.796042 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be72da9a-7085-443f-9aad-625394e05d2b service placement] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:02:15.799554 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f45d28-355c-4c21-b75f-b3f94a40ed8f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ba74e28ced4cbba857f54222dfbe41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:02:15.800144 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f45d28-355c-4c21-b75f-b3f94a40ed8f service cinder] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:02:15.800422 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f45d28-355c-4c21-b75f-b3f94a40ed8f service cinder] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:02:15.800693 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f45d28-355c-4c21-b75f-b3f94a40ed8f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:02:15.809694 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f45d28-355c-4c21-b75f-b3f94a40ed8f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:02:15.811103 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f45d28-355c-4c21-b75f-b3f94a40ed8f service cinder] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:02:15.819740 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1241/2492] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:02:15 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:02:25.815754 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=35 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:02:45.286386 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dd54b8-a191-4304-b283-4dc3de07f7a2 service nova] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:02:45.287892 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-84dd54b8-a191-4304-b283-4dc3de07f7a2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:02:45.340835 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dd54b8-a191-4304-b283-4dc3de07f7a2 service nova] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:02:45.346942 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56e63d9-3736-4e93-9c2e-bf07b1614ad1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a943008193914f76a663d489bd73e873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:02:45.348491 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56e63d9-3736-4e93-9c2e-bf07b1614ad1 service nova] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:02:45.348935 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56e63d9-3736-4e93-9c2e-bf07b1614ad1 service nova] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:02:45.349207 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56e63d9-3736-4e93-9c2e-bf07b1614ad1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:02:45.361728 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56e63d9-3736-4e93-9c2e-bf07b1614ad1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:02:45.363482 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56e63d9-3736-4e93-9c2e-bf07b1614ad1 service nova] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:02:45.374801 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1252/2493] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:02:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:02:45.643190 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f45d28-355c-4c21-b75f-b3f94a40ed8f service cinder] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:02:45.645049 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-74f45d28-355c-4c21-b75f-b3f94a40ed8f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:02:45.726778 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f45d28-355c-4c21-b75f-b3f94a40ed8f service cinder] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:02:45.730995 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990c6bfa-a443-4a75-a554-92310afb8b46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:02:45.731821 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990c6bfa-a443-4a75-a554-92310afb8b46 service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:02:45.732106 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990c6bfa-a443-4a75-a554-92310afb8b46 service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:02:45.732382 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990c6bfa-a443-4a75-a554-92310afb8b46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:02:45.734247 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-990c6bfa-a443-4a75-a554-92310afb8b46 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:02:45.748588 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990c6bfa-a443-4a75-a554-92310afb8b46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:02:45.750094 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990c6bfa-a443-4a75-a554-92310afb8b46 service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:02:45.761039 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1242/2494] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:02:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:02:48.235560 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56e63d9-3736-4e93-9c2e-bf07b1614ad1 service nova] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:02:48.244936 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56e63d9-3736-4e93-9c2e-bf07b1614ad1 service nova] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:02:48.249703 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de1026f-b790-483e-a86a-2240493ecec4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:02:48.250292 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de1026f-b790-483e-a86a-2240493ecec4 service neutron] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:02:48.250571 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de1026f-b790-483e-a86a-2240493ecec4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:02:48.250838 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de1026f-b790-483e-a86a-2240493ecec4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:02:48.252663 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-0de1026f-b790-483e-a86a-2240493ecec4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:02:48.265657 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de1026f-b790-483e-a86a-2240493ecec4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:02:48.267140 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de1026f-b790-483e-a86a-2240493ecec4 service neutron] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:02:48.278202 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1253/2495] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:02:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:02:55.756226 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=35 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:02:58.273353 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=17 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:03:36.617782 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990c6bfa-a443-4a75-a554-92310afb8b46 service neutron] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:03:36.619395 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-990c6bfa-a443-4a75-a554-92310afb8b46 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:03:36.630871 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990c6bfa-a443-4a75-a554-92310afb8b46 service neutron] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:03:36.634516 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b8c288-0b6b-4a97-a832-727ea5fe3127 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ba74e28ced4cbba857f54222dfbe41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:03:36.635113 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b8c288-0b6b-4a97-a832-727ea5fe3127 service cinder] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:03:36.635382 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b8c288-0b6b-4a97-a832-727ea5fe3127 service cinder] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:03:36.635653 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b8c288-0b6b-4a97-a832-727ea5fe3127 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:03:36.637686 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-36b8c288-0b6b-4a97-a832-727ea5fe3127 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:03:36.721548 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b8c288-0b6b-4a97-a832-727ea5fe3127 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:03:36.722974 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b8c288-0b6b-4a97-a832-727ea5fe3127 service cinder] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:03:36.741292 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1243/2496] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:03:36 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:03:46.738759 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=41 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:04:00.065166 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de1026f-b790-483e-a86a-2240493ecec4 service neutron] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:04:00.074885 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de1026f-b790-483e-a86a-2240493ecec4 service neutron] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:04:00.078514 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bd1caa-9ff3-4130-85d7-a63355308247 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:04:00.079112 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bd1caa-9ff3-4130-85d7-a63355308247 service neutron] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:04:00.079388 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bd1caa-9ff3-4130-85d7-a63355308247 service neutron] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:04:00.079652 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bd1caa-9ff3-4130-85d7-a63355308247 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:04:00.081594 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-a5bd1caa-9ff3-4130-85d7-a63355308247 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:04:00.092224 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bd1caa-9ff3-4130-85d7-a63355308247 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:04:00.093751 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bd1caa-9ff3-4130-85d7-a63355308247 service neutron] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:04:00.102100 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1254/2497] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:04:00 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:04:10.098372 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=6 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:05:37.021866 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b8c288-0b6b-4a97-a832-727ea5fe3127 service cinder] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:05:37.031235 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b8c288-0b6b-4a97-a832-727ea5fe3127 service cinder] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:05:37.034812 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca506672-f95a-4424-b815-74e13ee239de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:05:37.035403 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca506672-f95a-4424-b815-74e13ee239de service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:05:37.035682 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca506672-f95a-4424-b815-74e13ee239de service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:05:37.035955 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca506672-f95a-4424-b815-74e13ee239de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:05:37.037785 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-ca506672-f95a-4424-b815-74e13ee239de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:05:37.123336 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca506672-f95a-4424-b815-74e13ee239de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:05:37.124812 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca506672-f95a-4424-b815-74e13ee239de service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:05:37.144690 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1244/2498] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:05:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:05:40.499248 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bd1caa-9ff3-4130-85d7-a63355308247 service neutron] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:05:40.500704 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-a5bd1caa-9ff3-4130-85d7-a63355308247 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:05:40.533370 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bd1caa-9ff3-4130-85d7-a63355308247 service neutron] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:05:40.538621 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6383951-c1ab-4d32-bf2f-ceaff5ec8b38 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2912307c56594125b157f1cfdf7c96a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:05:40.539456 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6383951-c1ab-4d32-bf2f-ceaff5ec8b38 service placement] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:05:40.539735 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6383951-c1ab-4d32-bf2f-ceaff5ec8b38 service placement] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:05:40.540012 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6383951-c1ab-4d32-bf2f-ceaff5ec8b38 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:05:40.542487 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-b6383951-c1ab-4d32-bf2f-ceaff5ec8b38 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:05:40.554784 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6383951-c1ab-4d32-bf2f-ceaff5ec8b38 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:05:40.556645 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6383951-c1ab-4d32-bf2f-ceaff5ec8b38 service placement] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:05:40.567309 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1255/2499] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:05:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:05:40.648113 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca506672-f95a-4424-b815-74e13ee239de service neutron] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:05:40.650240 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-ca506672-f95a-4424-b815-74e13ee239de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:05:40.664162 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca506672-f95a-4424-b815-74e13ee239de service neutron] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:05:40.667234 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30b3cdb-f0dc-4336-9f12-10b1f918d892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:05:40.667863 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30b3cdb-f0dc-4336-9f12-10b1f918d892 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:05:40.668137 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30b3cdb-f0dc-4336-9f12-10b1f918d892 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:05:40.668406 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30b3cdb-f0dc-4336-9f12-10b1f918d892 admin admin] PATH_INFO: `/v3/users/3bf116fade674838b9219e15dd20eda2` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:05:40.669543 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30b3cdb-f0dc-4336-9f12-10b1f918d892 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3bf116fade674838b9219e15dd20eda2)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:05:40.671529 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30b3cdb-f0dc-4336-9f12-10b1f918d892 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:05:40.821758 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-b30b3cdb-f0dc-4336-9f12-10b1f918d892 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3bf116fade674838b9219e15dd20eda2'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:05:40.833806 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-b30b3cdb-f0dc-4336-9f12-10b1f918d892 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3bf116fade674838b9219e15dd20eda2'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:05:40.841564 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-b30b3cdb-f0dc-4336-9f12-10b1f918d892 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3bf116fade674838b9219e15dd20eda2'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:05:40.842245 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-b30b3cdb-f0dc-4336-9f12-10b1f918d892 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3bf116fade674838b9219e15dd20eda2'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:05:40.842850 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-b30b3cdb-f0dc-4336-9f12-10b1f918d892 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3bf116fade674838b9219e15dd20eda2'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:05:40.927677 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1245/2500] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 09:05:40 2022] DELETE /identity/v3/users/3bf116fade674838b9219e15dd20eda2 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:05:41.141778 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6383951-c1ab-4d32-bf2f-ceaff5ec8b38 service placement] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:05:41.152666 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6383951-c1ab-4d32-bf2f-ceaff5ec8b38 service placement] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:05:41.214032 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9839f348-af28-4b54-b055-648fd8258c0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:05:41.214657 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9839f348-af28-4b54-b055-648fd8258c0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:05:41.214970 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9839f348-af28-4b54-b055-648fd8258c0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:05:41.215237 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9839f348-af28-4b54-b055-648fd8258c0e admin admin] PATH_INFO: `/v3/projects/37950ef781584676ae470fd2c6394ae3` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:05:41.216348 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9839f348-af28-4b54-b055-648fd8258c0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=37950ef781584676ae470fd2c6394ae3)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:05:41.218279 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9839f348-af28-4b54-b055-648fd8258c0e admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:05:41.351509 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-9839f348-af28-4b54-b055-648fd8258c0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37950ef781584676ae470fd2c6394ae3', 'request_id': 'req-9839f348-af28-4b54-b055-648fd8258c0e'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:05:41.358044 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-9839f348-af28-4b54-b055-648fd8258c0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37950ef781584676ae470fd2c6394ae3', 'request_id': 'req-9839f348-af28-4b54-b055-648fd8258c0e'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:05:41.436844 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-9839f348-af28-4b54-b055-648fd8258c0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37950ef781584676ae470fd2c6394ae3', 'request_id': 'req-9839f348-af28-4b54-b055-648fd8258c0e'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:05:41.448852 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-9839f348-af28-4b54-b055-648fd8258c0e admin admin] The token cache is being invalidate because project 37950ef781584676ae470fd2c6394ae3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 09:05:41.449193 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-9839f348-af28-4b54-b055-648fd8258c0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:05:41.449854 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-9839f348-af28-4b54-b055-648fd8258c0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:05:41.451350 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1256/2501] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 09:05:41 2022] DELETE /identity/v3/projects/37950ef781584676ae470fd2c6394ae3 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:05:41.457246 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a41701b-c9a6-4494-9c54-c4a3131e004d None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:05:41.457535 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a41701b-c9a6-4494-9c54-c4a3131e004d None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:05:41.457861 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a41701b-c9a6-4494-9c54-c4a3131e004d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:05:41.512192 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:05:41.514982 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-2a41701b-c9a6-4494-9c54-c4a3131e004d None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:05:41.516176 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-2a41701b-c9a6-4494-9c54-c4a3131e004d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:05:41.586694 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1246/2502] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 09:05:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:05:41.596943 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cad00df-fcd6-44e2-93f5-4c89d0c88484 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:05:41.597507 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cad00df-fcd6-44e2-93f5-4c89d0c88484 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:05:41.597636 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cad00df-fcd6-44e2-93f5-4c89d0c88484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:05:41.619607 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-6cad00df-fcd6-44e2-93f5-4c89d0c88484 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:05:41.620725 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-6cad00df-fcd6-44e2-93f5-4c89d0c88484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:05:41.628749 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1257/2503] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 09:05:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:05:41.633436 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a41701b-c9a6-4494-9c54-c4a3131e004d None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:05:41.643807 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a41701b-c9a6-4494-9c54-c4a3131e004d None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:05:41.648521 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ea2fc2-e869-46c1-bd40-68e69762b07d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:05:41.649151 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ea2fc2-e869-46c1-bd40-68e69762b07d admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:05:41.649419 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ea2fc2-e869-46c1-bd40-68e69762b07d admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:05:41.650207 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ea2fc2-e869-46c1-bd40-68e69762b07d admin admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:05:41.650857 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ea2fc2-e869-46c1-bd40-68e69762b07d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:05:41.651288 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ea2fc2-e869-46c1-bd40-68e69762b07d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:05:41.652891 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ea2fc2-e869-46c1-bd40-68e69762b07d admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:05:41.666392 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1247/2504] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Aug 10 09:05:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:05:41.676820 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cad00df-fcd6-44e2-93f5-4c89d0c88484 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:05:41.686760 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cad00df-fcd6-44e2-93f5-4c89d0c88484 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:05:41.693226 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f293ab8-17a8-4781-8894-98bf9d980f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:05:41.693844 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f293ab8-17a8-4781-8894-98bf9d980f22 admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:05:41.694129 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f293ab8-17a8-4781-8894-98bf9d980f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:05:41.694402 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f293ab8-17a8-4781-8894-98bf9d980f22 admin admin] PATH_INFO: `/v3/projects` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:05:41.695030 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f293ab8-17a8-4781-8894-98bf9d980f22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:05:41.697428 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f293ab8-17a8-4781-8894-98bf9d980f22 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:05:41.800954 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1258/2505] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 09:05:41 2022] POST /identity/v3/projects => generated 405 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 09:05:41.802813 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ea2fc2-e869-46c1-bd40-68e69762b07d admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:05:41.813021 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ea2fc2-e869-46c1-bd40-68e69762b07d admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:05:41.868217 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6515fbb6-1c46-4777-9b63-1522a057d627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:05:41.868836 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6515fbb6-1c46-4777-9b63-1522a057d627 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:05:41.869224 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6515fbb6-1c46-4777-9b63-1522a057d627 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:05:41.869496 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6515fbb6-1c46-4777-9b63-1522a057d627 admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:05:41.870120 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6515fbb6-1c46-4777-9b63-1522a057d627 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:05:41.872020 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6515fbb6-1c46-4777-9b63-1522a057d627 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:05:41.956690 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1248/2506] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 09:05:41 2022] POST /identity/v3/users => generated 303 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 09:05:41.960904 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f293ab8-17a8-4781-8894-98bf9d980f22 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:05:41.971749 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f293ab8-17a8-4781-8894-98bf9d980f22 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:05:41.976270 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1fcf34-51e1-4d06-831f-900a84b85862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:05:41.976868 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1fcf34-51e1-4d06-831f-900a84b85862 admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:05:41.977180 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1fcf34-51e1-4d06-831f-900a84b85862 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:05:41.977453 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1fcf34-51e1-4d06-831f-900a84b85862 admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:05:41.978697 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1fcf34-51e1-4d06-831f-900a84b85862 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:05:41.980298 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1fcf34-51e1-4d06-831f-900a84b85862 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:05:41.990054 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1259/2507] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 09:05:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 09:05:41.993906 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6515fbb6-1c46-4777-9b63-1522a057d627 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:05:42.004571 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6515fbb6-1c46-4777-9b63-1522a057d627 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:05:42.009804 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d60d656-4a80-4eab-bd43-cc7041eb2d74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:05:42.010394 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d60d656-4a80-4eab-bd43-cc7041eb2d74 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:05:42.010670 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d60d656-4a80-4eab-bd43-cc7041eb2d74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:05:42.010938 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d60d656-4a80-4eab-bd43-cc7041eb2d74 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:05:42.011644 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d60d656-4a80-4eab-bd43-cc7041eb2d74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:05:42.013243 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d60d656-4a80-4eab-bd43-cc7041eb2d74 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:05:42.023846 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1249/2508] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 09:05:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 09:05:42.028378 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1fcf34-51e1-4d06-831f-900a84b85862 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:05:42.038204 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1fcf34-51e1-4d06-831f-900a84b85862 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:05:42.043404 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f21dd7e-bd70-4112-b3fb-63183b31f790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:05:42.044040 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f21dd7e-bd70-4112-b3fb-63183b31f790 admin admin] REQUEST_METHOD: `PUT` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:05:42.044321 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f21dd7e-bd70-4112-b3fb-63183b31f790 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:05:42.044597 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f21dd7e-bd70-4112-b3fb-63183b31f790 admin admin] PATH_INFO: `/v3/projects/18b39c2e9ed2448982dae2c4e4d864b1/users/1e50f77f78834dd285e25176f79043bb/roles/44b64db484f24a46bf0b7ced46b519bf` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:05:42.059201 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f21dd7e-bd70-4112-b3fb-63183b31f790 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18b39c2e9ed2448982dae2c4e4d864b1, user_id=1e50f77f78834dd285e25176f79043bb, role_id=44b64db484f24a46bf0b7ced46b519bf)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:05:42.061907 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f21dd7e-bd70-4112-b3fb-63183b31f790 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:05:42.064047 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:05:42.133476 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1260/2509] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 09:05:42 2022] PUT /identity/v3/projects/18b39c2e9ed2448982dae2c4e4d864b1/users/1e50f77f78834dd285e25176f79043bb/roles/44b64db484f24a46bf0b7ced46b519bf => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:05:42.144790 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f0345b-af9a-4d7f-a036-1d54d92eaa98 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:05:42.145127 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f0345b-af9a-4d7f-a036-1d54d92eaa98 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:05:42.145403 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f0345b-af9a-4d7f-a036-1d54d92eaa98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:05:42.173432 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-62f0345b-af9a-4d7f-a036-1d54d92eaa98 None None] MFA Rules not processed for user `1e50f77f78834dd285e25176f79043bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:05:42.174625 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-62f0345b-af9a-4d7f-a036-1d54d92eaa98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:05:42.237048 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1250/2510] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 09:05:42 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:05:42.241467 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f21dd7e-bd70-4112-b3fb-63183b31f790 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:05:42.303020 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f21dd7e-bd70-4112-b3fb-63183b31f790 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:05:42.307838 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa44d624-9f35-4dd8-9a44-e2f7f25f9e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:05:42.308438 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa44d624-9f35-4dd8-9a44-e2f7f25f9e4a admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:05:42.308717 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa44d624-9f35-4dd8-9a44-e2f7f25f9e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:05:42.309013 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa44d624-9f35-4dd8-9a44-e2f7f25f9e4a admin admin] PATH_INFO: `/v3/projects` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:05:42.309646 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa44d624-9f35-4dd8-9a44-e2f7f25f9e4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:05:42.311547 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa44d624-9f35-4dd8-9a44-e2f7f25f9e4a admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:05:42.433206 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1261/2511] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 09:05:42 2022] POST /identity/v3/projects => generated 405 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 09:05:42.437383 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f0345b-af9a-4d7f-a036-1d54d92eaa98 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:05:42.442841 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:05:42.449081 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f0345b-af9a-4d7f-a036-1d54d92eaa98 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:05:42.504846 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee465bb-1b61-42ef-8578-6ed4e18db6d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:05:42.505389 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee465bb-1b61-42ef-8578-6ed4e18db6d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:05:42.505668 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee465bb-1b61-42ef-8578-6ed4e18db6d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:05:42.505937 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee465bb-1b61-42ef-8578-6ed4e18db6d8 admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:05:42.506546 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee465bb-1b61-42ef-8578-6ed4e18db6d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:05:42.508477 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee465bb-1b61-42ef-8578-6ed4e18db6d8 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:05:42.562456 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1251/2512] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 09:05:42 2022] POST /identity/v3/users => generated 309 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 09:05:42.566624 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa44d624-9f35-4dd8-9a44-e2f7f25f9e4a admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:05:42.577893 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa44d624-9f35-4dd8-9a44-e2f7f25f9e4a admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:05:42.582531 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5b07ac-d70c-4319-903b-668bfd526d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:05:42.583123 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5b07ac-d70c-4319-903b-668bfd526d2c admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:05:42.583400 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5b07ac-d70c-4319-903b-668bfd526d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:05:42.583697 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5b07ac-d70c-4319-903b-668bfd526d2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:05:42.584328 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5b07ac-d70c-4319-903b-668bfd526d2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:05:42.585988 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5b07ac-d70c-4319-903b-668bfd526d2c admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:05:42.596479 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1262/2513] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 09:05:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 09:05:42.600527 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee465bb-1b61-42ef-8578-6ed4e18db6d8 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:05:42.611153 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee465bb-1b61-42ef-8578-6ed4e18db6d8 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:05:42.615679 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cc0f94-85b6-4d0b-a032-a5ca3ff76de2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:05:42.616310 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cc0f94-85b6-4d0b-a032-a5ca3ff76de2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:05:42.616587 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cc0f94-85b6-4d0b-a032-a5ca3ff76de2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:05:42.616856 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cc0f94-85b6-4d0b-a032-a5ca3ff76de2 admin admin] PATH_INFO: `/v3/projects/0b01ba524611411e9eac528e63da4e06/users/3fd3545f38db4b25994bc1b18f171b5d/roles/ec3751692d9c4ecf81b4a71c936c46be` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:05:42.632212 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cc0f94-85b6-4d0b-a032-a5ca3ff76de2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b01ba524611411e9eac528e63da4e06, user_id=3fd3545f38db4b25994bc1b18f171b5d, role_id=ec3751692d9c4ecf81b4a71c936c46be)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:05:42.636000 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cc0f94-85b6-4d0b-a032-a5ca3ff76de2 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:05:42.699597 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1252/2514] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 09:05:42 2022] PUT /identity/v3/projects/0b01ba524611411e9eac528e63da4e06/users/3fd3545f38db4b25994bc1b18f171b5d/roles/ec3751692d9c4ecf81b4a71c936c46be => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:05:42.703615 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5b07ac-d70c-4319-903b-668bfd526d2c admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:05:42.714379 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5b07ac-d70c-4319-903b-668bfd526d2c admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:05:42.772818 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee60b01-b07e-46a1-a59f-22865cc89b71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:05:42.773476 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee60b01-b07e-46a1-a59f-22865cc89b71 admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:05:42.773752 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee60b01-b07e-46a1-a59f-22865cc89b71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:05:42.774020 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee60b01-b07e-46a1-a59f-22865cc89b71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:05:42.774656 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee60b01-b07e-46a1-a59f-22865cc89b71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:05:42.776410 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee60b01-b07e-46a1-a59f-22865cc89b71 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:05:42.786851 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1263/2515] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 09:05:42 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 09:05:42.790862 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cc0f94-85b6-4d0b-a032-a5ca3ff76de2 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:05:42.801767 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cc0f94-85b6-4d0b-a032-a5ca3ff76de2 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:05:42.806394 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8e329a-ae3a-4396-b3a2-db88f97adedc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:05:42.807268 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8e329a-ae3a-4396-b3a2-db88f97adedc admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:05:42.808538 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8e329a-ae3a-4396-b3a2-db88f97adedc admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:05:42.808538 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8e329a-ae3a-4396-b3a2-db88f97adedc admin admin] PATH_INFO: `/v3/projects/0b01ba524611411e9eac528e63da4e06/users/3fd3545f38db4b25994bc1b18f171b5d/roles/ec3751692d9c4ecf81b4a71c936c46be` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:05:42.809879 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8e329a-ae3a-4396-b3a2-db88f97adedc admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b01ba524611411e9eac528e63da4e06, user_id=3fd3545f38db4b25994bc1b18f171b5d, role_id=ec3751692d9c4ecf81b4a71c936c46be)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:05:42.812501 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8e329a-ae3a-4396-b3a2-db88f97adedc admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:05:42.823298 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1253/2516] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 09:05:42 2022] PUT /identity/v3/projects/0b01ba524611411e9eac528e63da4e06/users/3fd3545f38db4b25994bc1b18f171b5d/roles/ec3751692d9c4ecf81b4a71c936c46be => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 09:05:42.833385 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd1ea1c-2a9b-4fda-a25d-cc6a1f8d2e3d None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:05:42.833671 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd1ea1c-2a9b-4fda-a25d-cc6a1f8d2e3d None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:05:42.833942 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd1ea1c-2a9b-4fda-a25d-cc6a1f8d2e3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:05:42.863375 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-0fd1ea1c-2a9b-4fda-a25d-cc6a1f8d2e3d None None] MFA Rules not processed for user `3fd3545f38db4b25994bc1b18f171b5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:05:42.864559 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-0fd1ea1c-2a9b-4fda-a25d-cc6a1f8d2e3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:05:42.913904 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:05:42.933156 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1264/2517] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 09:05:42 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:05:43.813715 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8e329a-ae3a-4396-b3a2-db88f97adedc admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:05:43.815433 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-fd8e329a-ae3a-4396-b3a2-db88f97adedc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:05:43.888196 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8e329a-ae3a-4396-b3a2-db88f97adedc admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:05:43.893970 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491567b6-3199-4a6d-bf0d-7f4d1010cbe6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a943008193914f76a663d489bd73e873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:05:43.894569 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491567b6-3199-4a6d-bf0d-7f4d1010cbe6 service nova] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:05:43.894893 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491567b6-3199-4a6d-bf0d-7f4d1010cbe6 service nova] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:05:43.895150 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491567b6-3199-4a6d-bf0d-7f4d1010cbe6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:05:43.896863 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-491567b6-3199-4a6d-bf0d-7f4d1010cbe6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:05:43.945034 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:05:43.955018 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491567b6-3199-4a6d-bf0d-7f4d1010cbe6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:05:43.956619 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491567b6-3199-4a6d-bf0d-7f4d1010cbe6 service nova] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:05:43.966562 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1254/2518] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:05:43 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:05:52.984842 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=5 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:05:54.064759 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=4 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:05:56.370619 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd1ea1c-2a9b-4fda-a25d-cc6a1f8d2e3d None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:05:56.372004 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-0fd1ea1c-2a9b-4fda-a25d-cc6a1f8d2e3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:05:56.439531 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd1ea1c-2a9b-4fda-a25d-cc6a1f8d2e3d None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:05:56.444713 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1400e929-ca78-47b6-bd43-2d346acacdc9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2912307c56594125b157f1cfdf7c96a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:05:56.445349 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1400e929-ca78-47b6-bd43-2d346acacdc9 service placement] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:05:56.445634 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1400e929-ca78-47b6-bd43-2d346acacdc9 service placement] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:05:56.445906 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1400e929-ca78-47b6-bd43-2d346acacdc9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:05:56.448180 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-1400e929-ca78-47b6-bd43-2d346acacdc9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:05:56.459445 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1400e929-ca78-47b6-bd43-2d346acacdc9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:05:56.461043 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1400e929-ca78-47b6-bd43-2d346acacdc9 service placement] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:05:56.470669 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1265/2519] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:05:56 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:06:06.466529 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=32 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:06:47.921767 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491567b6-3199-4a6d-bf0d-7f4d1010cbe6 service nova] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:06:47.923190 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-491567b6-3199-4a6d-bf0d-7f4d1010cbe6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:06:47.978200 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491567b6-3199-4a6d-bf0d-7f4d1010cbe6 service nova] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:06:47.982845 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b220c34-0299-4116-a20f-5aeb7da7cb7e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ba74e28ced4cbba857f54222dfbe41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:06:47.983453 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b220c34-0299-4116-a20f-5aeb7da7cb7e service cinder] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:06:47.983801 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b220c34-0299-4116-a20f-5aeb7da7cb7e service cinder] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:06:47.984075 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b220c34-0299-4116-a20f-5aeb7da7cb7e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:06:47.993518 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b220c34-0299-4116-a20f-5aeb7da7cb7e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:06:47.994954 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b220c34-0299-4116-a20f-5aeb7da7cb7e service cinder] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:06:48.004509 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1255/2520] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:06:47 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:06:58.000209 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=26 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:07:37.020377 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1400e929-ca78-47b6-bd43-2d346acacdc9 service placement] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:07:37.021865 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-1400e929-ca78-47b6-bd43-2d346acacdc9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:07:37.033669 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1400e929-ca78-47b6-bd43-2d346acacdc9 service placement] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:07:37.037349 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db285f6e-665f-4a8f-bbb2-8fb0e6553678 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ba74e28ced4cbba857f54222dfbe41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:07:37.040897 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db285f6e-665f-4a8f-bbb2-8fb0e6553678 service cinder] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:07:37.040897 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db285f6e-665f-4a8f-bbb2-8fb0e6553678 service cinder] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:07:37.040897 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db285f6e-665f-4a8f-bbb2-8fb0e6553678 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:07:37.040897 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-db285f6e-665f-4a8f-bbb2-8fb0e6553678 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:07:37.123796 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db285f6e-665f-4a8f-bbb2-8fb0e6553678 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:07:37.125282 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db285f6e-665f-4a8f-bbb2-8fb0e6553678 service cinder] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:07:37.144255 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1266/2521] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:07:37 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:07:37.188835 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b220c34-0299-4116-a20f-5aeb7da7cb7e service cinder] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:07:37.197835 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b220c34-0299-4116-a20f-5aeb7da7cb7e service cinder] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:07:37.201718 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f02279-20c7-4865-8b06-b29225975107 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ba74e28ced4cbba857f54222dfbe41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:07:37.202345 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f02279-20c7-4865-8b06-b29225975107 service cinder] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:07:37.202613 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f02279-20c7-4865-8b06-b29225975107 service cinder] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:07:37.202882 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f02279-20c7-4865-8b06-b29225975107 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:07:37.204575 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-49f02279-20c7-4865-8b06-b29225975107 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:07:37.289234 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f02279-20c7-4865-8b06-b29225975107 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:07:37.290592 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f02279-20c7-4865-8b06-b29225975107 service cinder] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:07:37.309463 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1256/2522] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:07:37 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:07:37.437887 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db285f6e-665f-4a8f-bbb2-8fb0e6553678 service cinder] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:07:37.499296 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db285f6e-665f-4a8f-bbb2-8fb0e6553678 service cinder] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:07:37.503825 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7421a6-225c-4d29-b898-82745f01528f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:07:37.504446 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7421a6-225c-4d29-b898-82745f01528f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:07:37.504729 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7421a6-225c-4d29-b898-82745f01528f admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:07:37.505032 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7421a6-225c-4d29-b898-82745f01528f admin admin] PATH_INFO: `/v3/users/1e50f77f78834dd285e25176f79043bb` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:07:37.506143 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7421a6-225c-4d29-b898-82745f01528f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e50f77f78834dd285e25176f79043bb)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:07:37.508058 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7421a6-225c-4d29-b898-82745f01528f admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:07:37.682165 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-3b7421a6-225c-4d29-b898-82745f01528f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e50f77f78834dd285e25176f79043bb'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:07:37.694295 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-3b7421a6-225c-4d29-b898-82745f01528f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e50f77f78834dd285e25176f79043bb'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:07:37.700217 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:07:37.702185 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-3b7421a6-225c-4d29-b898-82745f01528f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e50f77f78834dd285e25176f79043bb'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:07:37.757742 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-3b7421a6-225c-4d29-b898-82745f01528f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e50f77f78834dd285e25176f79043bb'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:07:37.758417 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-3b7421a6-225c-4d29-b898-82745f01528f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e50f77f78834dd285e25176f79043bb'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:07:37.765070 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1267/2523] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 09:07:37 2022] DELETE /identity/v3/users/1e50f77f78834dd285e25176f79043bb => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:07:37.770128 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2696574-9c5a-4ad7-b508-e7d6cd7b070d None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:07:37.770256 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2696574-9c5a-4ad7-b508-e7d6cd7b070d None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:07:37.770486 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2696574-9c5a-4ad7-b508-e7d6cd7b070d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:07:37.789721 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-e2696574-9c5a-4ad7-b508-e7d6cd7b070d None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:07:37.793137 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-e2696574-9c5a-4ad7-b508-e7d6cd7b070d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:07:37.849537 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1257/2524] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 09:07:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 09:07:37.861103 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7421a6-225c-4d29-b898-82745f01528f admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:07:37.862453 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-3b7421a6-225c-4d29-b898-82745f01528f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:07:37.919911 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7421a6-225c-4d29-b898-82745f01528f admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:07:37.923570 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4019265-8cc0-4ae9-bbf5-f366aad1ce2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:07:37.924170 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4019265-8cc0-4ae9-bbf5-f366aad1ce2f service neutron] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:07:37.924449 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4019265-8cc0-4ae9-bbf5-f366aad1ce2f service neutron] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:07:37.924721 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4019265-8cc0-4ae9-bbf5-f366aad1ce2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:07:37.934514 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4019265-8cc0-4ae9-bbf5-f366aad1ce2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:07:37.935935 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4019265-8cc0-4ae9-bbf5-f366aad1ce2f service neutron] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:07:37.945260 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1268/2525] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:07:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:07:38.326413 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2696574-9c5a-4ad7-b508-e7d6cd7b070d None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:07:38.327952 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-e2696574-9c5a-4ad7-b508-e7d6cd7b070d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:07:38.340705 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2696574-9c5a-4ad7-b508-e7d6cd7b070d None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:07:38.347038 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2911eaa9-f359-4e03-a754-37d7b0169c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:07:38.347667 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2911eaa9-f359-4e03-a754-37d7b0169c83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:07:38.347942 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2911eaa9-f359-4e03-a754-37d7b0169c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:07:38.348194 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2911eaa9-f359-4e03-a754-37d7b0169c83 admin admin] PATH_INFO: `/v3/projects/18b39c2e9ed2448982dae2c4e4d864b1` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:07:38.349446 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2911eaa9-f359-4e03-a754-37d7b0169c83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=18b39c2e9ed2448982dae2c4e4d864b1)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:07:38.352672 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2911eaa9-f359-4e03-a754-37d7b0169c83 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:07:38.451399 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-2911eaa9-f359-4e03-a754-37d7b0169c83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18b39c2e9ed2448982dae2c4e4d864b1', 'request_id': 'req-2911eaa9-f359-4e03-a754-37d7b0169c83'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:07:38.507617 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-2911eaa9-f359-4e03-a754-37d7b0169c83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18b39c2e9ed2448982dae2c4e4d864b1', 'request_id': 'req-2911eaa9-f359-4e03-a754-37d7b0169c83'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:07:38.514065 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-2911eaa9-f359-4e03-a754-37d7b0169c83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18b39c2e9ed2448982dae2c4e4d864b1', 'request_id': 'req-2911eaa9-f359-4e03-a754-37d7b0169c83'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:07:38.515236 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:07:38.526221 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-2911eaa9-f359-4e03-a754-37d7b0169c83 admin admin] The token cache is being invalidate because project 18b39c2e9ed2448982dae2c4e4d864b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 09:07:38.526984 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-2911eaa9-f359-4e03-a754-37d7b0169c83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:07:38.527651 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-2911eaa9-f359-4e03-a754-37d7b0169c83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:07:38.529175 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1258/2526] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 09:07:38 2022] DELETE /identity/v3/projects/18b39c2e9ed2448982dae2c4e4d864b1 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:07:38.533332 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4019265-8cc0-4ae9-bbf5-f366aad1ce2f service neutron] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:07:38.543519 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4019265-8cc0-4ae9-bbf5-f366aad1ce2f service neutron] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:07:38.598086 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bb7e65-1a15-4a91-a204-05da24caf045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:07:38.598712 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bb7e65-1a15-4a91-a204-05da24caf045 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:07:38.598996 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bb7e65-1a15-4a91-a204-05da24caf045 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:07:38.599267 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bb7e65-1a15-4a91-a204-05da24caf045 admin admin] PATH_INFO: `/v3/users/3fd3545f38db4b25994bc1b18f171b5d` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:07:38.600381 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bb7e65-1a15-4a91-a204-05da24caf045 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3fd3545f38db4b25994bc1b18f171b5d)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:07:38.602327 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bb7e65-1a15-4a91-a204-05da24caf045 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:07:38.748746 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-62bb7e65-1a15-4a91-a204-05da24caf045 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3fd3545f38db4b25994bc1b18f171b5d'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:07:38.762325 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-62bb7e65-1a15-4a91-a204-05da24caf045 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3fd3545f38db4b25994bc1b18f171b5d'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:07:38.769858 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-62bb7e65-1a15-4a91-a204-05da24caf045 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3fd3545f38db4b25994bc1b18f171b5d'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:07:38.826982 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-62bb7e65-1a15-4a91-a204-05da24caf045 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3fd3545f38db4b25994bc1b18f171b5d'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:07:38.827601 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-62bb7e65-1a15-4a91-a204-05da24caf045 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3fd3545f38db4b25994bc1b18f171b5d'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:07:38.834139 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1269/2527] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 09:07:38 2022] DELETE /identity/v3/users/3fd3545f38db4b25994bc1b18f171b5d => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:07:39.212089 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2911eaa9-f359-4e03-a754-37d7b0169c83 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:07:39.218265 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-2911eaa9-f359-4e03-a754-37d7b0169c83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:07:39.227685 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2911eaa9-f359-4e03-a754-37d7b0169c83 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:07:39.287899 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8669e14-f631-48d2-a39d-973f99a99e82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:07:39.288539 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8669e14-f631-48d2-a39d-973f99a99e82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:07:39.288820 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8669e14-f631-48d2-a39d-973f99a99e82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:07:39.289164 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8669e14-f631-48d2-a39d-973f99a99e82 admin admin] PATH_INFO: `/v3/projects/0b01ba524611411e9eac528e63da4e06` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:07:39.290500 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8669e14-f631-48d2-a39d-973f99a99e82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b01ba524611411e9eac528e63da4e06)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:07:39.293118 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8669e14-f631-48d2-a39d-973f99a99e82 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:07:39.469972 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-f8669e14-f631-48d2-a39d-973f99a99e82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b01ba524611411e9eac528e63da4e06', 'request_id': 'req-f8669e14-f631-48d2-a39d-973f99a99e82'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:07:39.536408 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-f8669e14-f631-48d2-a39d-973f99a99e82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b01ba524611411e9eac528e63da4e06', 'request_id': 'req-f8669e14-f631-48d2-a39d-973f99a99e82'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:07:39.543180 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-f8669e14-f631-48d2-a39d-973f99a99e82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b01ba524611411e9eac528e63da4e06', 'request_id': 'req-f8669e14-f631-48d2-a39d-973f99a99e82'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:07:39.554360 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-f8669e14-f631-48d2-a39d-973f99a99e82 admin admin] The token cache is being invalidate because project 0b01ba524611411e9eac528e63da4e06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 09:07:39.554646 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-f8669e14-f631-48d2-a39d-973f99a99e82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:07:39.555269 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-f8669e14-f631-48d2-a39d-973f99a99e82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:07:39.556752 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1259/2528] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 09:07:39 2022] DELETE /identity/v3/projects/0b01ba524611411e9eac528e63da4e06 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:07:39.562463 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38451f50-09c1-43f6-b0de-9c8999058abd None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:07:39.562957 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38451f50-09c1-43f6-b0de-9c8999058abd None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:07:39.563078 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38451f50-09c1-43f6-b0de-9c8999058abd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:07:39.582883 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-38451f50-09c1-43f6-b0de-9c8999058abd None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:07:39.584114 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-38451f50-09c1-43f6-b0de-9c8999058abd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:07:39.589449 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:07:39.642421 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1270/2529] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 09:07:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:07:39.652058 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e11ccc-1a43-46f7-a50e-40d466a8dc91 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:07:39.652346 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e11ccc-1a43-46f7-a50e-40d466a8dc91 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:07:39.652621 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e11ccc-1a43-46f7-a50e-40d466a8dc91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:07:39.672516 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-12e11ccc-1a43-46f7-a50e-40d466a8dc91 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:07:39.673739 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-12e11ccc-1a43-46f7-a50e-40d466a8dc91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:07:39.681609 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1260/2530] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 09:07:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:07:39.686488 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38451f50-09c1-43f6-b0de-9c8999058abd None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:07:39.696295 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38451f50-09c1-43f6-b0de-9c8999058abd None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:07:39.701504 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248f3cdb-f1e5-4475-a733-d9188037d6f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:07:39.702096 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248f3cdb-f1e5-4475-a733-d9188037d6f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:07:39.702373 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248f3cdb-f1e5-4475-a733-d9188037d6f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:07:39.702644 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248f3cdb-f1e5-4475-a733-d9188037d6f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:07:39.703284 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248f3cdb-f1e5-4475-a733-d9188037d6f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:07:39.703587 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248f3cdb-f1e5-4475-a733-d9188037d6f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:07:39.705208 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248f3cdb-f1e5-4475-a733-d9188037d6f4 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:07:39.716464 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1271/2531] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Aug 10 09:07:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:07:39.720676 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e11ccc-1a43-46f7-a50e-40d466a8dc91 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:07:39.729813 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e11ccc-1a43-46f7-a50e-40d466a8dc91 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:07:39.734993 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2214c24e-8346-4fc5-95fe-f50fbfcaa749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:07:39.735581 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2214c24e-8346-4fc5-95fe-f50fbfcaa749 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:07:39.735861 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2214c24e-8346-4fc5-95fe-f50fbfcaa749 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:07:39.736131 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2214c24e-8346-4fc5-95fe-f50fbfcaa749 admin admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:07:39.736766 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2214c24e-8346-4fc5-95fe-f50fbfcaa749 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:07:39.738727 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2214c24e-8346-4fc5-95fe-f50fbfcaa749 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:07:39.852714 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1261/2532] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 09:07:39 2022] POST /identity/v3/projects => generated 387 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 09:07:39.857526 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248f3cdb-f1e5-4475-a733-d9188037d6f4 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:07:39.867850 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248f3cdb-f1e5-4475-a733-d9188037d6f4 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:07:39.926895 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bc0234-367a-4803-a42c-bf5065f2329a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:07:39.927500 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bc0234-367a-4803-a42c-bf5065f2329a admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:07:39.927778 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bc0234-367a-4803-a42c-bf5065f2329a admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:07:39.928049 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bc0234-367a-4803-a42c-bf5065f2329a admin admin] PATH_INFO: `/v3/users` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:07:39.928657 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bc0234-367a-4803-a42c-bf5065f2329a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:07:39.930624 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bc0234-367a-4803-a42c-bf5065f2329a admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:07:40.009399 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1272/2533] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 09:07:39 2022] POST /identity/v3/users => generated 294 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 09:07:40.014137 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2214c24e-8346-4fc5-95fe-f50fbfcaa749 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:07:40.024458 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2214c24e-8346-4fc5-95fe-f50fbfcaa749 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:07:40.030100 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacf8edb-d613-472e-bf1a-5320171e4933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:07:40.033569 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacf8edb-d613-472e-bf1a-5320171e4933 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:07:40.033569 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacf8edb-d613-472e-bf1a-5320171e4933 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:07:40.033569 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacf8edb-d613-472e-bf1a-5320171e4933 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:07:40.033569 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacf8edb-d613-472e-bf1a-5320171e4933 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:07:40.033994 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacf8edb-d613-472e-bf1a-5320171e4933 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:07:40.044693 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1262/2534] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 09:07:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 09:07:40.048689 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bc0234-367a-4803-a42c-bf5065f2329a admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:07:40.058698 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bc0234-367a-4803-a42c-bf5065f2329a admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:07:40.063329 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341080ef-adb8-45d0-97df-c4988c89bc7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:07:40.063847 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341080ef-adb8-45d0-97df-c4988c89bc7d admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:07:40.064129 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341080ef-adb8-45d0-97df-c4988c89bc7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:07:40.064401 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341080ef-adb8-45d0-97df-c4988c89bc7d admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:07:40.065185 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341080ef-adb8-45d0-97df-c4988c89bc7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:07:40.066774 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341080ef-adb8-45d0-97df-c4988c89bc7d admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:07:40.077755 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1273/2535] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 09:07:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 09:07:40.081807 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacf8edb-d613-472e-bf1a-5320171e4933 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:07:40.092336 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacf8edb-d613-472e-bf1a-5320171e4933 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:07:40.096865 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30920097-b1cc-44dd-8a4b-305f6e84e8b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:07:40.097522 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30920097-b1cc-44dd-8a4b-305f6e84e8b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:07:40.097803 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30920097-b1cc-44dd-8a4b-305f6e84e8b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:07:40.098074 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30920097-b1cc-44dd-8a4b-305f6e84e8b2 admin admin] PATH_INFO: `/v3/projects/950519bedd3f4c91bfd56863632835ac/users/880df92c1b4d42b49fbf587cc3977379/roles/44b64db484f24a46bf0b7ced46b519bf` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:07:40.112033 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30920097-b1cc-44dd-8a4b-305f6e84e8b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=950519bedd3f4c91bfd56863632835ac, user_id=880df92c1b4d42b49fbf587cc3977379, role_id=44b64db484f24a46bf0b7ced46b519bf)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:07:40.114723 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30920097-b1cc-44dd-8a4b-305f6e84e8b2 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:07:40.193558 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1263/2536] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 09:07:40 2022] PUT /identity/v3/projects/950519bedd3f4c91bfd56863632835ac/users/880df92c1b4d42b49fbf587cc3977379/roles/44b64db484f24a46bf0b7ced46b519bf => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 09:07:40.204521 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceccdeb-eee0-414b-8dc4-09fcb32b0332 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:07:40.204805 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceccdeb-eee0-414b-8dc4-09fcb32b0332 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:07:40.205117 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceccdeb-eee0-414b-8dc4-09fcb32b0332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:07:40.231809 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-8ceccdeb-eee0-414b-8dc4-09fcb32b0332 None None] MFA Rules not processed for user `880df92c1b4d42b49fbf587cc3977379`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:07:40.233060 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-8ceccdeb-eee0-414b-8dc4-09fcb32b0332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:07:40.295907 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1274/2537] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 09:07:40 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:07:40.300455 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30920097-b1cc-44dd-8a4b-305f6e84e8b2 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:07:40.322883 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:07:40.367058 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30920097-b1cc-44dd-8a4b-305f6e84e8b2 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:07:40.372302 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00450d37-f624-4332-a7c2-a738be5d1521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:07:40.372901 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00450d37-f624-4332-a7c2-a738be5d1521 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:07:40.373208 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00450d37-f624-4332-a7c2-a738be5d1521 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:07:40.373480 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00450d37-f624-4332-a7c2-a738be5d1521 admin admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:07:40.374105 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00450d37-f624-4332-a7c2-a738be5d1521 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:07:40.376003 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00450d37-f624-4332-a7c2-a738be5d1521 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:07:40.480569 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1264/2538] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 09:07:40 2022] POST /identity/v3/projects => generated 387 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 09:07:40.484505 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceccdeb-eee0-414b-8dc4-09fcb32b0332 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:07:40.496862 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceccdeb-eee0-414b-8dc4-09fcb32b0332 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:07:40.500616 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:07:40.554742 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baf59af-3336-4f46-9061-9c63e501317c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:07:40.555349 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baf59af-3336-4f46-9061-9c63e501317c admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:07:40.555630 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baf59af-3336-4f46-9061-9c63e501317c admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:07:40.555900 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baf59af-3336-4f46-9061-9c63e501317c admin admin] PATH_INFO: `/v3/users` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:07:40.556515 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baf59af-3336-4f46-9061-9c63e501317c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:07:40.559108 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baf59af-3336-4f46-9061-9c63e501317c admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:07:40.668043 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1275/2539] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 09:07:40 2022] POST /identity/v3/users => generated 300 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 09:07:40.672200 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00450d37-f624-4332-a7c2-a738be5d1521 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:07:40.682518 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00450d37-f624-4332-a7c2-a738be5d1521 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:07:40.687714 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff55fe7-021a-47a9-8921-34e6ed09dbf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:07:40.688311 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff55fe7-021a-47a9-8921-34e6ed09dbf2 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:07:40.688591 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff55fe7-021a-47a9-8921-34e6ed09dbf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:07:40.688861 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff55fe7-021a-47a9-8921-34e6ed09dbf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:07:40.689524 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff55fe7-021a-47a9-8921-34e6ed09dbf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:07:40.691113 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff55fe7-021a-47a9-8921-34e6ed09dbf2 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:07:40.703915 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1265/2540] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 09:07:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 09:07:40.705584 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baf59af-3336-4f46-9061-9c63e501317c admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:07:40.715405 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baf59af-3336-4f46-9061-9c63e501317c admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:07:40.720093 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503bb2e5-6412-4511-a338-91b39a33af5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:07:40.720767 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503bb2e5-6412-4511-a338-91b39a33af5a admin admin] REQUEST_METHOD: `PUT` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:07:40.721081 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503bb2e5-6412-4511-a338-91b39a33af5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:07:40.721355 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503bb2e5-6412-4511-a338-91b39a33af5a admin admin] PATH_INFO: `/v3/projects/e6206428de3a457c891afbef2d98a1b0/users/f76b31ce9aa345b49d9b85272b9ebc57/roles/ec3751692d9c4ecf81b4a71c936c46be` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:07:40.736665 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503bb2e5-6412-4511-a338-91b39a33af5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6206428de3a457c891afbef2d98a1b0, user_id=f76b31ce9aa345b49d9b85272b9ebc57, role_id=ec3751692d9c4ecf81b4a71c936c46be)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:07:40.739363 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503bb2e5-6412-4511-a338-91b39a33af5a admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:07:40.811147 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1276/2541] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 09:07:40 2022] PUT /identity/v3/projects/e6206428de3a457c891afbef2d98a1b0/users/f76b31ce9aa345b49d9b85272b9ebc57/roles/ec3751692d9c4ecf81b4a71c936c46be => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:07:40.815121 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff55fe7-021a-47a9-8921-34e6ed09dbf2 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:07:40.825459 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff55fe7-021a-47a9-8921-34e6ed09dbf2 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:07:40.881149 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf78471-fb4b-4dcb-8ee2-9fa893e72927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:07:40.881747 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf78471-fb4b-4dcb-8ee2-9fa893e72927 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:07:40.882028 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf78471-fb4b-4dcb-8ee2-9fa893e72927 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:07:40.882309 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf78471-fb4b-4dcb-8ee2-9fa893e72927 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:07:40.882939 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf78471-fb4b-4dcb-8ee2-9fa893e72927 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:07:40.884532 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf78471-fb4b-4dcb-8ee2-9fa893e72927 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:07:40.894466 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1266/2542] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 09:07:40 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 09:07:40.898512 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503bb2e5-6412-4511-a338-91b39a33af5a admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:07:40.909566 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503bb2e5-6412-4511-a338-91b39a33af5a admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:07:40.914169 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8962a6e-7406-465d-b30b-565d630fe9b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:07:40.914809 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8962a6e-7406-465d-b30b-565d630fe9b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:07:40.915086 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8962a6e-7406-465d-b30b-565d630fe9b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:07:40.915355 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8962a6e-7406-465d-b30b-565d630fe9b4 admin admin] PATH_INFO: `/v3/projects/e6206428de3a457c891afbef2d98a1b0/users/f76b31ce9aa345b49d9b85272b9ebc57/roles/ec3751692d9c4ecf81b4a71c936c46be` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:07:40.918472 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8962a6e-7406-465d-b30b-565d630fe9b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6206428de3a457c891afbef2d98a1b0, user_id=f76b31ce9aa345b49d9b85272b9ebc57, role_id=ec3751692d9c4ecf81b4a71c936c46be)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:07:40.921059 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8962a6e-7406-465d-b30b-565d630fe9b4 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:07:40.931799 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1277/2543] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 09:07:40 2022] PUT /identity/v3/projects/e6206428de3a457c891afbef2d98a1b0/users/f76b31ce9aa345b49d9b85272b9ebc57/roles/ec3751692d9c4ecf81b4a71c936c46be => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:07:40.942186 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56002ebc-a430-4a6b-93e4-8521514bde7a None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:07:40.942475 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56002ebc-a430-4a6b-93e4-8521514bde7a None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:07:40.942752 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56002ebc-a430-4a6b-93e4-8521514bde7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:07:40.971663 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-56002ebc-a430-4a6b-93e4-8521514bde7a None None] MFA Rules not processed for user `f76b31ce9aa345b49d9b85272b9ebc57`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:07:40.972875 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-56002ebc-a430-4a6b-93e4-8521514bde7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:07:41.043217 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1267/2544] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 09:07:40 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:07:41.054424 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8962a6e-7406-465d-b30b-565d630fe9b4 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:07:41.056119 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-a8962a6e-7406-465d-b30b-565d630fe9b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:07:41.113138 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8962a6e-7406-465d-b30b-565d630fe9b4 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:07:41.116753 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1ad22a-14ca-4da2-99e0-56e94a70f78b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ba74e28ced4cbba857f54222dfbe41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:07:41.117381 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1ad22a-14ca-4da2-99e0-56e94a70f78b service cinder] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:07:41.117663 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1ad22a-14ca-4da2-99e0-56e94a70f78b service cinder] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:07:41.117935 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1ad22a-14ca-4da2-99e0-56e94a70f78b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:07:41.127406 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1ad22a-14ca-4da2-99e0-56e94a70f78b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:07:41.128832 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1ad22a-14ca-4da2-99e0-56e94a70f78b service cinder] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:07:41.138740 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1278/2545] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:07:41 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:07:41.202981 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56002ebc-a430-4a6b-93e4-8521514bde7a None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:07:41.221103 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:07:41.265409 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56002ebc-a430-4a6b-93e4-8521514bde7a None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:07:41.270201 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5b897d-57ab-4540-9034-45a206b4fd37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:07:41.270829 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5b897d-57ab-4540-9034-45a206b4fd37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:07:41.271106 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5b897d-57ab-4540-9034-45a206b4fd37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:07:41.271376 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5b897d-57ab-4540-9034-45a206b4fd37 admin admin] PATH_INFO: `/v3/users/880df92c1b4d42b49fbf587cc3977379` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:07:41.272617 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5b897d-57ab-4540-9034-45a206b4fd37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=880df92c1b4d42b49fbf587cc3977379)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:07:41.274572 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5b897d-57ab-4540-9034-45a206b4fd37 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:07:41.413830 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-fc5b897d-57ab-4540-9034-45a206b4fd37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '880df92c1b4d42b49fbf587cc3977379'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:07:41.426594 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-fc5b897d-57ab-4540-9034-45a206b4fd37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '880df92c1b4d42b49fbf587cc3977379'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:07:41.434381 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-fc5b897d-57ab-4540-9034-45a206b4fd37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '880df92c1b4d42b49fbf587cc3977379'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:07:41.435045 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-fc5b897d-57ab-4540-9034-45a206b4fd37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '880df92c1b4d42b49fbf587cc3977379'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:07:41.435666 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-fc5b897d-57ab-4540-9034-45a206b4fd37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '880df92c1b4d42b49fbf587cc3977379'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:07:41.502208 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1268/2546] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 09:07:41 2022] DELETE /identity/v3/users/880df92c1b4d42b49fbf587cc3977379 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:07:41.506666 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0853e8a-01bf-4b6e-8915-cff32189204c None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:07:41.506949 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0853e8a-01bf-4b6e-8915-cff32189204c None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:07:41.507225 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0853e8a-01bf-4b6e-8915-cff32189204c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:07:41.525615 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-e0853e8a-01bf-4b6e-8915-cff32189204c None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:07:41.526935 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-e0853e8a-01bf-4b6e-8915-cff32189204c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:07:41.585594 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1279/2547] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 09:07:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:07:41.597841 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5b897d-57ab-4540-9034-45a206b4fd37 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:07:41.599252 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-fc5b897d-57ab-4540-9034-45a206b4fd37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:07:41.656812 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5b897d-57ab-4540-9034-45a206b4fd37 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:07:41.660450 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5da9e36-3a55-446d-9eb3-b49433d97145 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:07:41.661080 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5da9e36-3a55-446d-9eb3-b49433d97145 service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:07:41.661360 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5da9e36-3a55-446d-9eb3-b49433d97145 service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:07:41.661630 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5da9e36-3a55-446d-9eb3-b49433d97145 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:07:41.671276 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5da9e36-3a55-446d-9eb3-b49433d97145 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:07:41.672696 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5da9e36-3a55-446d-9eb3-b49433d97145 service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:07:41.682184 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1269/2548] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:07:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:07:42.099373 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0853e8a-01bf-4b6e-8915-cff32189204c None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:07:42.101394 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-e0853e8a-01bf-4b6e-8915-cff32189204c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:07:42.109313 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:07:42.115480 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0853e8a-01bf-4b6e-8915-cff32189204c None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:07:42.120765 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b63579-d7a4-494f-bafc-cfc9e15432a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:07:42.121216 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b63579-d7a4-494f-bafc-cfc9e15432a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:07:42.121562 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b63579-d7a4-494f-bafc-cfc9e15432a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:07:42.121895 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b63579-d7a4-494f-bafc-cfc9e15432a1 admin admin] PATH_INFO: `/v3/projects/950519bedd3f4c91bfd56863632835ac` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:07:42.123096 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b63579-d7a4-494f-bafc-cfc9e15432a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=950519bedd3f4c91bfd56863632835ac)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:07:42.125126 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b63579-d7a4-494f-bafc-cfc9e15432a1 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:07:42.244398 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-c5b63579-d7a4-494f-bafc-cfc9e15432a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '950519bedd3f4c91bfd56863632835ac', 'request_id': 'req-c5b63579-d7a4-494f-bafc-cfc9e15432a1'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:07:42.249983 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-c5b63579-d7a4-494f-bafc-cfc9e15432a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '950519bedd3f4c91bfd56863632835ac', 'request_id': 'req-c5b63579-d7a4-494f-bafc-cfc9e15432a1'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:07:42.301875 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-c5b63579-d7a4-494f-bafc-cfc9e15432a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '950519bedd3f4c91bfd56863632835ac', 'request_id': 'req-c5b63579-d7a4-494f-bafc-cfc9e15432a1'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:07:42.313125 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-c5b63579-d7a4-494f-bafc-cfc9e15432a1 admin admin] The token cache is being invalidate because project 950519bedd3f4c91bfd56863632835ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 09:07:42.313413 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-c5b63579-d7a4-494f-bafc-cfc9e15432a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:07:42.314069 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-c5b63579-d7a4-494f-bafc-cfc9e15432a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:07:42.315547 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1280/2549] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 09:07:42 2022] DELETE /identity/v3/projects/950519bedd3f4c91bfd56863632835ac => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:07:42.319603 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5da9e36-3a55-446d-9eb3-b49433d97145 service neutron] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:07:42.330319 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5da9e36-3a55-446d-9eb3-b49433d97145 service neutron] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:07:42.364223 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:07:42.387502 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fb2f39-ac37-446a-bed0-d05837e8037a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:07:42.388204 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fb2f39-ac37-446a-bed0-d05837e8037a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:07:42.388611 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fb2f39-ac37-446a-bed0-d05837e8037a admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:07:42.389369 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fb2f39-ac37-446a-bed0-d05837e8037a admin admin] PATH_INFO: `/v3/users/f76b31ce9aa345b49d9b85272b9ebc57` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:07:42.390626 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fb2f39-ac37-446a-bed0-d05837e8037a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f76b31ce9aa345b49d9b85272b9ebc57)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:07:42.392854 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fb2f39-ac37-446a-bed0-d05837e8037a admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:07:42.548157 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-27fb2f39-ac37-446a-bed0-d05837e8037a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f76b31ce9aa345b49d9b85272b9ebc57'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:07:42.560504 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-27fb2f39-ac37-446a-bed0-d05837e8037a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f76b31ce9aa345b49d9b85272b9ebc57'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:07:42.567579 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-27fb2f39-ac37-446a-bed0-d05837e8037a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f76b31ce9aa345b49d9b85272b9ebc57'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:07:42.568242 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-27fb2f39-ac37-446a-bed0-d05837e8037a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f76b31ce9aa345b49d9b85272b9ebc57'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:07:42.568847 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-27fb2f39-ac37-446a-bed0-d05837e8037a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f76b31ce9aa345b49d9b85272b9ebc57'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:07:42.641035 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1270/2550] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 09:07:42 2022] DELETE /identity/v3/users/f76b31ce9aa345b49d9b85272b9ebc57 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 09:07:43.050962 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b63579-d7a4-494f-bafc-cfc9e15432a1 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:07:43.052418 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-c5b63579-d7a4-494f-bafc-cfc9e15432a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:07:43.067268 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b63579-d7a4-494f-bafc-cfc9e15432a1 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:07:43.124829 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8dbf93-589b-4d24-bb1b-7249b570d0c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:07:43.125529 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8dbf93-589b-4d24-bb1b-7249b570d0c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:07:43.125813 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8dbf93-589b-4d24-bb1b-7249b570d0c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:07:43.126089 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8dbf93-589b-4d24-bb1b-7249b570d0c5 admin admin] PATH_INFO: `/v3/projects/e6206428de3a457c891afbef2d98a1b0` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:07:43.127235 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8dbf93-589b-4d24-bb1b-7249b570d0c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6206428de3a457c891afbef2d98a1b0)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:07:43.129190 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8dbf93-589b-4d24-bb1b-7249b570d0c5 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:07:43.240223 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-8d8dbf93-589b-4d24-bb1b-7249b570d0c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6206428de3a457c891afbef2d98a1b0', 'request_id': 'req-8d8dbf93-589b-4d24-bb1b-7249b570d0c5'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:07:43.245822 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-8d8dbf93-589b-4d24-bb1b-7249b570d0c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6206428de3a457c891afbef2d98a1b0', 'request_id': 'req-8d8dbf93-589b-4d24-bb1b-7249b570d0c5'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:07:43.289982 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-8d8dbf93-589b-4d24-bb1b-7249b570d0c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6206428de3a457c891afbef2d98a1b0', 'request_id': 'req-8d8dbf93-589b-4d24-bb1b-7249b570d0c5'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:07:43.301667 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-8d8dbf93-589b-4d24-bb1b-7249b570d0c5 admin admin] The token cache is being invalidate because project e6206428de3a457c891afbef2d98a1b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 09:07:43.301889 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-8d8dbf93-589b-4d24-bb1b-7249b570d0c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:07:43.302505 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-8d8dbf93-589b-4d24-bb1b-7249b570d0c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:07:43.304031 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1281/2551] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 09:07:43 2022] DELETE /identity/v3/projects/e6206428de3a457c891afbef2d98a1b0 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:07:43.309074 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f434115-a181-46f3-95af-862a78defd73 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:07:43.309387 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f434115-a181-46f3-95af-862a78defd73 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:07:43.309664 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f434115-a181-46f3-95af-862a78defd73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:07:43.329454 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-6f434115-a181-46f3-95af-862a78defd73 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:07:43.330649 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-6f434115-a181-46f3-95af-862a78defd73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:07:43.388576 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1271/2552] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 09:07:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:07:43.451934 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb4d65f-7d35-42df-9878-a2d9e1dc397d None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:07:43.452221 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb4d65f-7d35-42df-9878-a2d9e1dc397d None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:07:43.452496 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb4d65f-7d35-42df-9878-a2d9e1dc397d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:07:43.473062 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-efb4d65f-7d35-42df-9878-a2d9e1dc397d None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:07:43.474340 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-efb4d65f-7d35-42df-9878-a2d9e1dc397d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:07:43.483140 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1282/2553] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 09:07:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:07:43.487529 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f434115-a181-46f3-95af-862a78defd73 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:07:43.497270 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f434115-a181-46f3-95af-862a78defd73 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:07:43.502056 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa451e71-4082-4bb4-b5e3-1088205449ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:07:43.502652 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa451e71-4082-4bb4-b5e3-1088205449ae admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:07:43.502929 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa451e71-4082-4bb4-b5e3-1088205449ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:07:43.503197 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa451e71-4082-4bb4-b5e3-1088205449ae admin admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:07:43.503842 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa451e71-4082-4bb4-b5e3-1088205449ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:07:43.504144 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa451e71-4082-4bb4-b5e3-1088205449ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:07:43.505757 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa451e71-4082-4bb4-b5e3-1088205449ae admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:07:43.518435 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1272/2554] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Aug 10 09:07:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:07:43.522570 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb4d65f-7d35-42df-9878-a2d9e1dc397d None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:07:43.532507 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb4d65f-7d35-42df-9878-a2d9e1dc397d None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:07:43.537356 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88315187-4a4d-4e7e-92e9-d900e2d1861d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:07:43.537954 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88315187-4a4d-4e7e-92e9-d900e2d1861d admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:07:43.538364 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88315187-4a4d-4e7e-92e9-d900e2d1861d admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:07:43.538640 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88315187-4a4d-4e7e-92e9-d900e2d1861d admin admin] PATH_INFO: `/v3/projects` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:07:43.539292 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88315187-4a4d-4e7e-92e9-d900e2d1861d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:07:43.541222 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88315187-4a4d-4e7e-92e9-d900e2d1861d admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:07:43.546695 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:07:43.675494 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1283/2555] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 09:07:43 2022] POST /identity/v3/projects => generated 377 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 09:07:43.679656 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa451e71-4082-4bb4-b5e3-1088205449ae admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:07:43.689915 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa451e71-4082-4bb4-b5e3-1088205449ae admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:07:43.731626 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:07:43.745853 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c93d404-3ede-4fc6-b2a2-267cf46138bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:07:43.746453 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c93d404-3ede-4fc6-b2a2-267cf46138bd admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:07:43.746727 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c93d404-3ede-4fc6-b2a2-267cf46138bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:07:43.746997 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c93d404-3ede-4fc6-b2a2-267cf46138bd admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:07:43.747618 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c93d404-3ede-4fc6-b2a2-267cf46138bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:07:43.749557 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c93d404-3ede-4fc6-b2a2-267cf46138bd admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:07:43.845215 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1273/2556] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 09:07:43 2022] POST /identity/v3/users => generated 289 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 09:07:43.849413 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88315187-4a4d-4e7e-92e9-d900e2d1861d admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:07:43.860391 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88315187-4a4d-4e7e-92e9-d900e2d1861d admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:07:43.865099 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77de61ee-a8d3-47aa-a7fe-06ccdcbcd40b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:07:43.865723 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77de61ee-a8d3-47aa-a7fe-06ccdcbcd40b admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:07:43.866004 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77de61ee-a8d3-47aa-a7fe-06ccdcbcd40b admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:07:43.866275 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77de61ee-a8d3-47aa-a7fe-06ccdcbcd40b admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:07:43.866902 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77de61ee-a8d3-47aa-a7fe-06ccdcbcd40b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:07:43.868497 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77de61ee-a8d3-47aa-a7fe-06ccdcbcd40b admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:07:43.878883 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1284/2557] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 09:07:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 09:07:43.882720 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c93d404-3ede-4fc6-b2a2-267cf46138bd admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:07:43.893534 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c93d404-3ede-4fc6-b2a2-267cf46138bd admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:07:43.898086 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269d8d21-d902-462e-875a-270268969f19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:07:43.898674 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269d8d21-d902-462e-875a-270268969f19 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:07:43.898952 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269d8d21-d902-462e-875a-270268969f19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:07:43.899224 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269d8d21-d902-462e-875a-270268969f19 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:07:43.899861 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269d8d21-d902-462e-875a-270268969f19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:07:43.901531 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269d8d21-d902-462e-875a-270268969f19 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:07:43.911841 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1274/2558] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 09:07:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 09:07:43.915872 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77de61ee-a8d3-47aa-a7fe-06ccdcbcd40b admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:07:43.925558 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77de61ee-a8d3-47aa-a7fe-06ccdcbcd40b admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:07:43.930127 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda2156d-5fab-4864-b287-b7a0d273ff8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:07:43.930767 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda2156d-5fab-4864-b287-b7a0d273ff8c admin admin] REQUEST_METHOD: `PUT` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:07:43.931046 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda2156d-5fab-4864-b287-b7a0d273ff8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:07:43.931319 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda2156d-5fab-4864-b287-b7a0d273ff8c admin admin] PATH_INFO: `/v3/projects/77587add00b24a3ba6d739ca829dd2f6/users/4e061a3b18b44a2e991b236b37c3f01c/roles/44b64db484f24a46bf0b7ced46b519bf` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:07:43.946023 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda2156d-5fab-4864-b287-b7a0d273ff8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=77587add00b24a3ba6d739ca829dd2f6, user_id=4e061a3b18b44a2e991b236b37c3f01c, role_id=44b64db484f24a46bf0b7ced46b519bf)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:07:43.949884 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda2156d-5fab-4864-b287-b7a0d273ff8c admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:07:44.010290 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1285/2559] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 09:07:43 2022] PUT /identity/v3/projects/77587add00b24a3ba6d739ca829dd2f6/users/4e061a3b18b44a2e991b236b37c3f01c/roles/44b64db484f24a46bf0b7ced46b519bf => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:07:44.021231 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6075c13-db28-4c48-a987-4437c0207772 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:07:44.021519 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6075c13-db28-4c48-a987-4437c0207772 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:07:44.021791 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6075c13-db28-4c48-a987-4437c0207772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:07:44.048901 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-a6075c13-db28-4c48-a987-4437c0207772 None None] MFA Rules not processed for user `4e061a3b18b44a2e991b236b37c3f01c`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:07:44.050137 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-a6075c13-db28-4c48-a987-4437c0207772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:07:44.113808 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1275/2560] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 09:07:44 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:07:44.118268 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda2156d-5fab-4864-b287-b7a0d273ff8c admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:07:44.179624 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda2156d-5fab-4864-b287-b7a0d273ff8c admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:07:44.185239 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f5001c-4149-4c1c-9639-4ced1ee50613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:07:44.185846 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f5001c-4149-4c1c-9639-4ced1ee50613 admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:07:44.186120 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f5001c-4149-4c1c-9639-4ced1ee50613 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:07:44.186439 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f5001c-4149-4c1c-9639-4ced1ee50613 admin admin] PATH_INFO: `/v3/projects` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:07:44.187048 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f5001c-4149-4c1c-9639-4ced1ee50613 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:07:44.188956 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f5001c-4149-4c1c-9639-4ced1ee50613 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:07:44.286219 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1286/2561] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 09:07:44 2022] POST /identity/v3/projects => generated 377 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 09:07:44.290434 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6075c13-db28-4c48-a987-4437c0207772 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:07:44.301988 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6075c13-db28-4c48-a987-4437c0207772 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:07:44.360945 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5455c314-2b2e-4ba9-bebf-d55de2b3dc70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:07:44.361561 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5455c314-2b2e-4ba9-bebf-d55de2b3dc70 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:07:44.361841 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5455c314-2b2e-4ba9-bebf-d55de2b3dc70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:07:44.362115 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5455c314-2b2e-4ba9-bebf-d55de2b3dc70 admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:07:44.362729 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5455c314-2b2e-4ba9-bebf-d55de2b3dc70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:07:44.364747 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5455c314-2b2e-4ba9-bebf-d55de2b3dc70 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:07:44.468359 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1276/2562] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 09:07:44 2022] POST /identity/v3/users => generated 289 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 09:07:44.472313 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f5001c-4149-4c1c-9639-4ced1ee50613 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:07:44.483338 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f5001c-4149-4c1c-9639-4ced1ee50613 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:07:44.488011 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd0d128-4b26-423d-852a-c0418b07a7e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:07:44.488619 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd0d128-4b26-423d-852a-c0418b07a7e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:07:44.488958 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd0d128-4b26-423d-852a-c0418b07a7e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:07:44.489224 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd0d128-4b26-423d-852a-c0418b07a7e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:07:44.489859 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd0d128-4b26-423d-852a-c0418b07a7e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:07:44.491483 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd0d128-4b26-423d-852a-c0418b07a7e8 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:07:44.502597 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1287/2563] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 09:07:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 09:07:44.506493 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5455c314-2b2e-4ba9-bebf-d55de2b3dc70 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:07:44.516896 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5455c314-2b2e-4ba9-bebf-d55de2b3dc70 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:07:44.521861 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542b2e1e-6e03-4b85-bb2f-24bde56fc3c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:07:44.522459 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542b2e1e-6e03-4b85-bb2f-24bde56fc3c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:07:44.522743 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542b2e1e-6e03-4b85-bb2f-24bde56fc3c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:07:44.523016 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542b2e1e-6e03-4b85-bb2f-24bde56fc3c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:07:44.523692 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542b2e1e-6e03-4b85-bb2f-24bde56fc3c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:07:44.525432 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542b2e1e-6e03-4b85-bb2f-24bde56fc3c0 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:07:44.537885 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1277/2564] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 09:07:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 09:07:44.545579 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd0d128-4b26-423d-852a-c0418b07a7e8 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:07:44.555757 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd0d128-4b26-423d-852a-c0418b07a7e8 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:07:44.561404 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d87609-769c-4407-889c-a4dcc81d6cf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:07:44.562052 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d87609-769c-4407-889c-a4dcc81d6cf7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:07:44.562337 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d87609-769c-4407-889c-a4dcc81d6cf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:07:44.562612 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d87609-769c-4407-889c-a4dcc81d6cf7 admin admin] PATH_INFO: `/v3/projects/b8c6410322774d1f8cf8b3a0cadc19fb/users/37e71e5d74684e2eb18b1b2e9f9c90ec/roles/44b64db484f24a46bf0b7ced46b519bf` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:07:44.577320 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d87609-769c-4407-889c-a4dcc81d6cf7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8c6410322774d1f8cf8b3a0cadc19fb, user_id=37e71e5d74684e2eb18b1b2e9f9c90ec, role_id=44b64db484f24a46bf0b7ced46b519bf)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:07:44.579983 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d87609-769c-4407-889c-a4dcc81d6cf7 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:07:44.644630 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1288/2565] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 09:07:44 2022] PUT /identity/v3/projects/b8c6410322774d1f8cf8b3a0cadc19fb/users/37e71e5d74684e2eb18b1b2e9f9c90ec/roles/44b64db484f24a46bf0b7ced46b519bf => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:07:44.659982 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52f5fd6-40cf-4da8-9b89-47aafd252497 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:07:44.660298 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52f5fd6-40cf-4da8-9b89-47aafd252497 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:07:44.660590 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52f5fd6-40cf-4da8-9b89-47aafd252497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:07:44.688271 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-e52f5fd6-40cf-4da8-9b89-47aafd252497 None None] MFA Rules not processed for user `37e71e5d74684e2eb18b1b2e9f9c90ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:07:44.689620 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-e52f5fd6-40cf-4da8-9b89-47aafd252497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:07:44.707772 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:07:44.756334 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1278/2566] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 09:07:44 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:07:44.761482 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d87609-769c-4407-889c-a4dcc81d6cf7 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:07:44.822941 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d87609-769c-4407-889c-a4dcc81d6cf7 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:07:44.833832 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7d532a-c925-4119-8449-ad70e5913496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:07:44.834449 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7d532a-c925-4119-8449-ad70e5913496 admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:07:44.834729 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7d532a-c925-4119-8449-ad70e5913496 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:07:44.835001 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7d532a-c925-4119-8449-ad70e5913496 admin admin] PATH_INFO: `/v3/projects` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:07:44.835621 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7d532a-c925-4119-8449-ad70e5913496 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:07:44.837575 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7d532a-c925-4119-8449-ad70e5913496 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:07:44.843666 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:07:44.948585 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1289/2567] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 09:07:44 2022] POST /identity/v3/projects => generated 379 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 09:07:44.952849 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52f5fd6-40cf-4da8-9b89-47aafd252497 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:07:44.963371 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52f5fd6-40cf-4da8-9b89-47aafd252497 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:07:45.019385 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b53372a-81e2-4806-a2d1-b31f33b5874d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:07:45.019886 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b53372a-81e2-4806-a2d1-b31f33b5874d admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:07:45.020165 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b53372a-81e2-4806-a2d1-b31f33b5874d admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:07:45.020435 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b53372a-81e2-4806-a2d1-b31f33b5874d admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:07:45.021089 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b53372a-81e2-4806-a2d1-b31f33b5874d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:07:45.022995 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b53372a-81e2-4806-a2d1-b31f33b5874d admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:07:45.124051 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1279/2568] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 09:07:44 2022] POST /identity/v3/users => generated 296 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 09:07:45.128196 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7d532a-c925-4119-8449-ad70e5913496 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:07:45.138801 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7d532a-c925-4119-8449-ad70e5913496 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:07:45.143735 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced3e27b-d3c8-4b7f-a807-5dbd3e0bf5c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:07:45.144329 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced3e27b-d3c8-4b7f-a807-5dbd3e0bf5c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:07:45.144613 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced3e27b-d3c8-4b7f-a807-5dbd3e0bf5c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:07:45.144890 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced3e27b-d3c8-4b7f-a807-5dbd3e0bf5c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:07:45.145560 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced3e27b-d3c8-4b7f-a807-5dbd3e0bf5c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:07:45.147160 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced3e27b-d3c8-4b7f-a807-5dbd3e0bf5c4 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:07:45.157079 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1290/2569] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 09:07:45 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 09:07:45.162441 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b53372a-81e2-4806-a2d1-b31f33b5874d admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:07:45.173024 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b53372a-81e2-4806-a2d1-b31f33b5874d admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:07:45.177608 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e89f3d5-a5a4-44d5-a143-92416bf54bfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:07:45.178252 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e89f3d5-a5a4-44d5-a143-92416bf54bfc admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:07:45.178536 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e89f3d5-a5a4-44d5-a143-92416bf54bfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:07:45.178812 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e89f3d5-a5a4-44d5-a143-92416bf54bfc admin admin] PATH_INFO: `/v3/projects/91c7b18993904001b076559f9bcdb89f/users/10d533f02e6a4f6184dfaf8525101d8b/roles/ec3751692d9c4ecf81b4a71c936c46be` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:07:45.194975 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e89f3d5-a5a4-44d5-a143-92416bf54bfc admin admin] RBAC: Authorizing `identity:create_grant(project_id=91c7b18993904001b076559f9bcdb89f, user_id=10d533f02e6a4f6184dfaf8525101d8b, role_id=ec3751692d9c4ecf81b4a71c936c46be)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:07:45.197692 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e89f3d5-a5a4-44d5-a143-92416bf54bfc admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:07:45.266346 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1280/2570] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 09:07:45 2022] PUT /identity/v3/projects/91c7b18993904001b076559f9bcdb89f/users/10d533f02e6a4f6184dfaf8525101d8b/roles/ec3751692d9c4ecf81b4a71c936c46be => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 09:07:45.274780 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced3e27b-d3c8-4b7f-a807-5dbd3e0bf5c4 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:07:45.280702 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced3e27b-d3c8-4b7f-a807-5dbd3e0bf5c4 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:07:45.336503 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009b5afc-faa6-46e8-bd67-428196de8c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:07:45.337136 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009b5afc-faa6-46e8-bd67-428196de8c89 admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:07:45.337415 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009b5afc-faa6-46e8-bd67-428196de8c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:07:45.337683 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009b5afc-faa6-46e8-bd67-428196de8c89 admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:07:45.338316 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009b5afc-faa6-46e8-bd67-428196de8c89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:07:45.340000 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009b5afc-faa6-46e8-bd67-428196de8c89 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:07:45.352345 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1291/2571] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 09:07:45 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 09:07:45.354055 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e89f3d5-a5a4-44d5-a143-92416bf54bfc admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:07:45.365188 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e89f3d5-a5a4-44d5-a143-92416bf54bfc admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:07:45.369776 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5bfca1-2b74-4081-aa87-1a5839d91225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:07:45.371213 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5bfca1-2b74-4081-aa87-1a5839d91225 admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:07:45.371492 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5bfca1-2b74-4081-aa87-1a5839d91225 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:07:45.371880 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5bfca1-2b74-4081-aa87-1a5839d91225 admin admin] PATH_INFO: `/v3/projects/91c7b18993904001b076559f9bcdb89f/users/10d533f02e6a4f6184dfaf8525101d8b/roles/ec3751692d9c4ecf81b4a71c936c46be` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:07:45.373831 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5bfca1-2b74-4081-aa87-1a5839d91225 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91c7b18993904001b076559f9bcdb89f, user_id=10d533f02e6a4f6184dfaf8525101d8b, role_id=ec3751692d9c4ecf81b4a71c936c46be)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:07:45.376439 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5bfca1-2b74-4081-aa87-1a5839d91225 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:07:45.387590 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1281/2572] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 09:07:45 2022] PUT /identity/v3/projects/91c7b18993904001b076559f9bcdb89f/users/10d533f02e6a4f6184dfaf8525101d8b/roles/ec3751692d9c4ecf81b4a71c936c46be => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:07:45.397498 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3f119a-714f-4b3f-a69e-e7fcf8cd97c2 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:07:45.397786 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3f119a-714f-4b3f-a69e-e7fcf8cd97c2 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:07:45.398092 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3f119a-714f-4b3f-a69e-e7fcf8cd97c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:07:45.428464 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-4c3f119a-714f-4b3f-a69e-e7fcf8cd97c2 None None] MFA Rules not processed for user `10d533f02e6a4f6184dfaf8525101d8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:07:45.429721 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-4c3f119a-714f-4b3f-a69e-e7fcf8cd97c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:07:45.495914 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1292/2573] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 09:07:45 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:07:45.507019 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5bfca1-2b74-4081-aa87-1a5839d91225 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:07:45.508699 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-7d5bfca1-2b74-4081-aa87-1a5839d91225 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:07:45.566184 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5bfca1-2b74-4081-aa87-1a5839d91225 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:07:45.569737 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbff13e6-7116-4696-a554-9be79a4b2ee9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ba74e28ced4cbba857f54222dfbe41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:07:45.570334 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbff13e6-7116-4696-a554-9be79a4b2ee9 service cinder] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:07:45.570613 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbff13e6-7116-4696-a554-9be79a4b2ee9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:07:45.570886 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbff13e6-7116-4696-a554-9be79a4b2ee9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:07:45.580326 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbff13e6-7116-4696-a554-9be79a4b2ee9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:07:45.581808 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbff13e6-7116-4696-a554-9be79a4b2ee9 service cinder] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:07:45.591718 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1282/2574] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:07:45 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:07:45.725992 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3f119a-714f-4b3f-a69e-e7fcf8cd97c2 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:07:45.736293 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3f119a-714f-4b3f-a69e-e7fcf8cd97c2 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:07:45.739894 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92e6968-b5e0-4e5d-8b82-c249d1541699 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ba74e28ced4cbba857f54222dfbe41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:07:45.740512 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92e6968-b5e0-4e5d-8b82-c249d1541699 service cinder] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:07:45.740795 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92e6968-b5e0-4e5d-8b82-c249d1541699 service cinder] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:07:45.741105 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92e6968-b5e0-4e5d-8b82-c249d1541699 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:07:45.795507 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:07:45.798026 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92e6968-b5e0-4e5d-8b82-c249d1541699 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:07:45.799453 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92e6968-b5e0-4e5d-8b82-c249d1541699 service cinder] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:07:45.809287 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1293/2575] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:07:45 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:07:46.489344 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbff13e6-7116-4696-a554-9be79a4b2ee9 service cinder] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:07:46.490802 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-dbff13e6-7116-4696-a554-9be79a4b2ee9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:07:46.509013 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:07:46.546694 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbff13e6-7116-4696-a554-9be79a4b2ee9 service cinder] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:07:46.550228 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490f3ee2-9f8e-4136-9b22-770f8edec96f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:07:46.550824 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490f3ee2-9f8e-4136-9b22-770f8edec96f service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:07:46.551102 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490f3ee2-9f8e-4136-9b22-770f8edec96f service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:07:46.551373 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490f3ee2-9f8e-4136-9b22-770f8edec96f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:07:46.553118 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-490f3ee2-9f8e-4136-9b22-770f8edec96f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:07:46.631896 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490f3ee2-9f8e-4136-9b22-770f8edec96f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:07:46.633444 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490f3ee2-9f8e-4136-9b22-770f8edec96f service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:07:46.645495 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1283/2576] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:07:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:07:49.953782 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92e6968-b5e0-4e5d-8b82-c249d1541699 service cinder] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:07:49.963004 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92e6968-b5e0-4e5d-8b82-c249d1541699 service cinder] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:07:49.967356 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fc3c65-112f-4be8-9b5e-6e0384d2f4a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:07:49.967931 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fc3c65-112f-4be8-9b5e-6e0384d2f4a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:07:49.968204 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fc3c65-112f-4be8-9b5e-6e0384d2f4a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:07:49.968469 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fc3c65-112f-4be8-9b5e-6e0384d2f4a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:07:49.970142 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-09fc3c65-112f-4be8-9b5e-6e0384d2f4a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:07:49.984067 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fc3c65-112f-4be8-9b5e-6e0384d2f4a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:07:49.985529 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fc3c65-112f-4be8-9b5e-6e0384d2f4a1 service neutron] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:07:49.996214 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1294/2577] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:07:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:07:56.656500 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=47 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:07:59.991450 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=8 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:09:05.044515 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490f3ee2-9f8e-4136-9b22-770f8edec96f service neutron] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:09:05.053269 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490f3ee2-9f8e-4136-9b22-770f8edec96f service neutron] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:09:05.057983 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35290daf-490e-4ae6-8f4d-3b4f52dcdc9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:09:05.058580 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35290daf-490e-4ae6-8f4d-3b4f52dcdc9e service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:09:05.058861 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35290daf-490e-4ae6-8f4d-3b4f52dcdc9e service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:09:05.059133 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35290daf-490e-4ae6-8f4d-3b4f52dcdc9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:09:05.060866 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-35290daf-490e-4ae6-8f4d-3b4f52dcdc9e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:09:05.070424 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35290daf-490e-4ae6-8f4d-3b4f52dcdc9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:09:05.073282 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35290daf-490e-4ae6-8f4d-3b4f52dcdc9e service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:09:05.080451 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1284/2578] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:09:05 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:09:15.076650 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=6 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:09:39.390713 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fc3c65-112f-4be8-9b5e-6e0384d2f4a1 service neutron] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:09:39.392551 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-09fc3c65-112f-4be8-9b5e-6e0384d2f4a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:09:39.408390 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fc3c65-112f-4be8-9b5e-6e0384d2f4a1 service neutron] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:09:39.415097 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fd4be2-7b69-44c2-be6d-0828cd2648a1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a943008193914f76a663d489bd73e873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:09:39.415618 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fd4be2-7b69-44c2-be6d-0828cd2648a1 service nova] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:09:39.415893 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fd4be2-7b69-44c2-be6d-0828cd2648a1 service nova] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:09:39.416159 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fd4be2-7b69-44c2-be6d-0828cd2648a1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:09:39.417994 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-19fd4be2-7b69-44c2-be6d-0828cd2648a1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:09:39.435875 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fd4be2-7b69-44c2-be6d-0828cd2648a1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:09:39.437862 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fd4be2-7b69-44c2-be6d-0828cd2648a1 service nova] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:09:39.460212 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1295/2579] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:09:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:09:49.447149 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=6 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:10:48.237483 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35290daf-490e-4ae6-8f4d-3b4f52dcdc9e service neutron] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:10:48.238899 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-35290daf-490e-4ae6-8f4d-3b4f52dcdc9e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:10:48.270566 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35290daf-490e-4ae6-8f4d-3b4f52dcdc9e service neutron] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:10:48.275359 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c624024-caee-4040-a93e-554dc009b87f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ba74e28ced4cbba857f54222dfbe41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:10:48.275966 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c624024-caee-4040-a93e-554dc009b87f service cinder] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:10:48.276247 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c624024-caee-4040-a93e-554dc009b87f service cinder] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:10:48.276518 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c624024-caee-4040-a93e-554dc009b87f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:10:48.278292 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-2c624024-caee-4040-a93e-554dc009b87f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:10:48.371632 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c624024-caee-4040-a93e-554dc009b87f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:10:48.373086 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c624024-caee-4040-a93e-554dc009b87f service cinder] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:10:48.391931 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1285/2580] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:10:48 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:10:58.389049 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=55 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:10:59.362548 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fd4be2-7b69-44c2-be6d-0828cd2648a1 service nova] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:10:59.364033 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-19fd4be2-7b69-44c2-be6d-0828cd2648a1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:10:59.488734 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fd4be2-7b69-44c2-be6d-0828cd2648a1 service nova] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:10:59.493920 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fb76c6-3bf1-47af-a7c0-38f120429a6b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2912307c56594125b157f1cfdf7c96a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:10:59.494528 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fb76c6-3bf1-47af-a7c0-38f120429a6b service placement] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:10:59.494804 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fb76c6-3bf1-47af-a7c0-38f120429a6b service placement] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:10:59.495089 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fb76c6-3bf1-47af-a7c0-38f120429a6b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:10:59.496785 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-05fb76c6-3bf1-47af-a7c0-38f120429a6b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:10:59.507979 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fb76c6-3bf1-47af-a7c0-38f120429a6b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:10:59.509582 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fb76c6-3bf1-47af-a7c0-38f120429a6b service placement] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:10:59.519414 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1296/2581] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:10:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:11:09.515113 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=54 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:12:37.566816 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c624024-caee-4040-a93e-554dc009b87f service cinder] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:12:37.606755 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c624024-caee-4040-a93e-554dc009b87f service cinder] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:12:37.612270 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80a0dd6-e19c-4fc3-9d4c-2dd71fb58938 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a943008193914f76a663d489bd73e873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:12:37.612868 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80a0dd6-e19c-4fc3-9d4c-2dd71fb58938 service nova] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:12:37.613178 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80a0dd6-e19c-4fc3-9d4c-2dd71fb58938 service nova] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:12:37.613454 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80a0dd6-e19c-4fc3-9d4c-2dd71fb58938 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:12:37.615187 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-d80a0dd6-e19c-4fc3-9d4c-2dd71fb58938 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:12:37.683000 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80a0dd6-e19c-4fc3-9d4c-2dd71fb58938 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:12:37.684588 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80a0dd6-e19c-4fc3-9d4c-2dd71fb58938 service nova] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:12:37.694578 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1286/2582] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:12:37 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:12:47.646542 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fb76c6-3bf1-47af-a7c0-38f120429a6b service placement] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:12:47.665429 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fb76c6-3bf1-47af-a7c0-38f120429a6b service placement] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:12:47.668974 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfd707d-81f8-46ef-8c49-e3cb19476d4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:12:47.669562 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfd707d-81f8-46ef-8c49-e3cb19476d4d service neutron] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:12:47.669840 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfd707d-81f8-46ef-8c49-e3cb19476d4d service neutron] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:12:47.670108 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfd707d-81f8-46ef-8c49-e3cb19476d4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:12:47.681049 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfd707d-81f8-46ef-8c49-e3cb19476d4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:12:47.682471 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfd707d-81f8-46ef-8c49-e3cb19476d4d service neutron] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:12:47.689275 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=44 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:12:47.693524 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1297/2583] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:12:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:12:48.646938 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80a0dd6-e19c-4fc3-9d4c-2dd71fb58938 service nova] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:12:48.656472 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80a0dd6-e19c-4fc3-9d4c-2dd71fb58938 service nova] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:12:48.659977 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10673ed5-f27b-48ae-b7fc-ccc4f463f105 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ba74e28ced4cbba857f54222dfbe41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:12:48.661132 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10673ed5-f27b-48ae-b7fc-ccc4f463f105 service cinder] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:12:48.661409 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10673ed5-f27b-48ae-b7fc-ccc4f463f105 service cinder] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:12:48.661711 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10673ed5-f27b-48ae-b7fc-ccc4f463f105 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:12:48.663416 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-10673ed5-f27b-48ae-b7fc-ccc4f463f105 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:12:48.725377 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10673ed5-f27b-48ae-b7fc-ccc4f463f105 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:12:48.726827 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10673ed5-f27b-48ae-b7fc-ccc4f463f105 service cinder] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:12:48.736494 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1287/2584] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:12:48 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:12:48.823783 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfd707d-81f8-46ef-8c49-e3cb19476d4d service neutron] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:12:48.834238 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfd707d-81f8-46ef-8c49-e3cb19476d4d service neutron] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:12:48.837890 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4041119-81b1-48cc-b5a3-04d1c92a14fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ba74e28ced4cbba857f54222dfbe41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:12:48.838490 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4041119-81b1-48cc-b5a3-04d1c92a14fc service cinder] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:12:48.838770 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4041119-81b1-48cc-b5a3-04d1c92a14fc service cinder] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:12:48.839045 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4041119-81b1-48cc-b5a3-04d1c92a14fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:12:48.849906 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4041119-81b1-48cc-b5a3-04d1c92a14fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:12:48.851345 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4041119-81b1-48cc-b5a3-04d1c92a14fc service cinder] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:12:48.860970 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1298/2585] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:12:48 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:12:51.009314 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10673ed5-f27b-48ae-b7fc-ccc4f463f105 service cinder] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:12:51.018902 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10673ed5-f27b-48ae-b7fc-ccc4f463f105 service cinder] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:12:51.022783 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e010de19-8ace-487f-a733-800a74708daa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:12:51.023382 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e010de19-8ace-487f-a733-800a74708daa service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:12:51.023660 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e010de19-8ace-487f-a733-800a74708daa service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:12:51.023928 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e010de19-8ace-487f-a733-800a74708daa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:12:51.034189 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e010de19-8ace-487f-a733-800a74708daa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:12:51.035615 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e010de19-8ace-487f-a733-800a74708daa service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:12:51.047107 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1288/2586] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:12:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:12:58.856705 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=16 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:13:01.042166 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=34 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:13:46.176132 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4041119-81b1-48cc-b5a3-04d1c92a14fc service cinder] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:13:46.185069 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4041119-81b1-48cc-b5a3-04d1c92a14fc service cinder] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:13:46.188787 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05d8955-a2b6-49fb-bb9a-477e5d1e9afd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ba74e28ced4cbba857f54222dfbe41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:13:46.189420 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05d8955-a2b6-49fb-bb9a-477e5d1e9afd service cinder] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:13:46.189815 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05d8955-a2b6-49fb-bb9a-477e5d1e9afd service cinder] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:13:46.190091 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05d8955-a2b6-49fb-bb9a-477e5d1e9afd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:13:46.199892 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05d8955-a2b6-49fb-bb9a-477e5d1e9afd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:13:46.201361 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05d8955-a2b6-49fb-bb9a-477e5d1e9afd service cinder] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:13:46.212385 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1299/2587] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:13:46 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:13:46.992511 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e010de19-8ace-487f-a733-800a74708daa service neutron] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:13:47.054311 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e010de19-8ace-487f-a733-800a74708daa service neutron] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:13:47.058881 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a589f6-8537-435b-95c7-62e3abb1ac4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:13:47.059501 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a589f6-8537-435b-95c7-62e3abb1ac4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:13:47.059779 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a589f6-8537-435b-95c7-62e3abb1ac4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:13:47.060052 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a589f6-8537-435b-95c7-62e3abb1ac4a admin admin] PATH_INFO: `/v3/users/4e061a3b18b44a2e991b236b37c3f01c` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:13:47.061188 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a589f6-8537-435b-95c7-62e3abb1ac4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e061a3b18b44a2e991b236b37c3f01c)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:13:47.063112 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a589f6-8537-435b-95c7-62e3abb1ac4a admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:13:47.202628 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-c9a589f6-8537-435b-95c7-62e3abb1ac4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e061a3b18b44a2e991b236b37c3f01c'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:13:47.215747 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-c9a589f6-8537-435b-95c7-62e3abb1ac4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e061a3b18b44a2e991b236b37c3f01c'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:13:47.223438 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-c9a589f6-8537-435b-95c7-62e3abb1ac4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e061a3b18b44a2e991b236b37c3f01c'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:13:47.224167 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-c9a589f6-8537-435b-95c7-62e3abb1ac4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e061a3b18b44a2e991b236b37c3f01c'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:13:47.224784 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-c9a589f6-8537-435b-95c7-62e3abb1ac4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e061a3b18b44a2e991b236b37c3f01c'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:13:47.281026 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1289/2588] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 09:13:46 2022] DELETE /identity/v3/users/4e061a3b18b44a2e991b236b37c3f01c => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:13:47.285498 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caef192c-c1fa-4c1a-a044-c7f4ba00c3a5 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:13:47.285962 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caef192c-c1fa-4c1a-a044-c7f4ba00c3a5 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:13:47.286080 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caef192c-c1fa-4c1a-a044-c7f4ba00c3a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:13:47.304720 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-caef192c-c1fa-4c1a-a044-c7f4ba00c3a5 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:13:47.305948 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-caef192c-c1fa-4c1a-a044-c7f4ba00c3a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:13:47.363414 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1300/2589] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 09:13:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:13:47.374474 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a589f6-8537-435b-95c7-62e3abb1ac4a admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:13:47.375849 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-c9a589f6-8537-435b-95c7-62e3abb1ac4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:13:47.448637 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a589f6-8537-435b-95c7-62e3abb1ac4a admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:13:47.452289 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eed66f3-8c56-4372-bed5-22477750d1ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:13:47.452886 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eed66f3-8c56-4372-bed5-22477750d1ed service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:13:47.453192 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eed66f3-8c56-4372-bed5-22477750d1ed service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:13:47.453463 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eed66f3-8c56-4372-bed5-22477750d1ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:13:47.462450 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eed66f3-8c56-4372-bed5-22477750d1ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:13:47.463867 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eed66f3-8c56-4372-bed5-22477750d1ed service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:13:47.473113 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1290/2590] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:13:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:13:47.826388 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caef192c-c1fa-4c1a-a044-c7f4ba00c3a5 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:13:47.827942 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-caef192c-c1fa-4c1a-a044-c7f4ba00c3a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:13:47.839874 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caef192c-c1fa-4c1a-a044-c7f4ba00c3a5 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:13:47.844584 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed29e72f-1eea-491a-b413-71848dc829af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:13:47.845302 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed29e72f-1eea-491a-b413-71848dc829af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:13:47.845582 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed29e72f-1eea-491a-b413-71848dc829af admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:13:47.846320 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed29e72f-1eea-491a-b413-71848dc829af admin admin] PATH_INFO: `/v3/projects/77587add00b24a3ba6d739ca829dd2f6` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:13:47.847708 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed29e72f-1eea-491a-b413-71848dc829af admin admin] RBAC: Authorizing `identity:delete_project(project_id=77587add00b24a3ba6d739ca829dd2f6)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:13:47.855256 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed29e72f-1eea-491a-b413-71848dc829af admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:13:47.953652 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-ed29e72f-1eea-491a-b413-71848dc829af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77587add00b24a3ba6d739ca829dd2f6', 'request_id': 'req-ed29e72f-1eea-491a-b413-71848dc829af'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:13:47.959696 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-ed29e72f-1eea-491a-b413-71848dc829af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77587add00b24a3ba6d739ca829dd2f6', 'request_id': 'req-ed29e72f-1eea-491a-b413-71848dc829af'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:13:48.035250 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-ed29e72f-1eea-491a-b413-71848dc829af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77587add00b24a3ba6d739ca829dd2f6', 'request_id': 'req-ed29e72f-1eea-491a-b413-71848dc829af'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:13:48.047482 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-ed29e72f-1eea-491a-b413-71848dc829af admin admin] The token cache is being invalidate because project 77587add00b24a3ba6d739ca829dd2f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 09:13:48.047772 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-ed29e72f-1eea-491a-b413-71848dc829af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:13:48.048398 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-ed29e72f-1eea-491a-b413-71848dc829af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:13:48.049906 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1301/2591] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 09:13:47 2022] DELETE /identity/v3/projects/77587add00b24a3ba6d739ca829dd2f6 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:13:48.054422 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eed66f3-8c56-4372-bed5-22477750d1ed service neutron] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:13:48.064220 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eed66f3-8c56-4372-bed5-22477750d1ed service neutron] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:13:48.072605 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:13:48.122005 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b7d04e-5240-4fc6-a49e-7e7a1233d70d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:13:48.122877 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b7d04e-5240-4fc6-a49e-7e7a1233d70d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:13:48.122877 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b7d04e-5240-4fc6-a49e-7e7a1233d70d admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:13:48.123173 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b7d04e-5240-4fc6-a49e-7e7a1233d70d admin admin] PATH_INFO: `/v3/users/37e71e5d74684e2eb18b1b2e9f9c90ec` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:13:48.124285 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b7d04e-5240-4fc6-a49e-7e7a1233d70d admin admin] RBAC: Authorizing `identity:delete_user(user_id=37e71e5d74684e2eb18b1b2e9f9c90ec)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:13:48.126246 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b7d04e-5240-4fc6-a49e-7e7a1233d70d admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:13:48.265554 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-98b7d04e-5240-4fc6-a49e-7e7a1233d70d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37e71e5d74684e2eb18b1b2e9f9c90ec'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:13:48.278020 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-98b7d04e-5240-4fc6-a49e-7e7a1233d70d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37e71e5d74684e2eb18b1b2e9f9c90ec'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:13:48.284812 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-98b7d04e-5240-4fc6-a49e-7e7a1233d70d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37e71e5d74684e2eb18b1b2e9f9c90ec'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:13:48.285520 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-98b7d04e-5240-4fc6-a49e-7e7a1233d70d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37e71e5d74684e2eb18b1b2e9f9c90ec'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:13:48.286134 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-98b7d04e-5240-4fc6-a49e-7e7a1233d70d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37e71e5d74684e2eb18b1b2e9f9c90ec'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:13:48.389399 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1291/2592] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 09:13:48 2022] DELETE /identity/v3/users/37e71e5d74684e2eb18b1b2e9f9c90ec => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:13:48.865599 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed29e72f-1eea-491a-b413-71848dc829af admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:13:48.867221 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-ed29e72f-1eea-491a-b413-71848dc829af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:13:48.881446 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed29e72f-1eea-491a-b413-71848dc829af admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:13:48.941353 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8778b0d7-3fb5-41ab-96c6-e69c8d004271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:13:48.941979 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8778b0d7-3fb5-41ab-96c6-e69c8d004271 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:13:48.942328 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8778b0d7-3fb5-41ab-96c6-e69c8d004271 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:13:48.942600 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8778b0d7-3fb5-41ab-96c6-e69c8d004271 admin admin] PATH_INFO: `/v3/projects/b8c6410322774d1f8cf8b3a0cadc19fb` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:13:48.944675 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8778b0d7-3fb5-41ab-96c6-e69c8d004271 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8c6410322774d1f8cf8b3a0cadc19fb)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:13:48.946649 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8778b0d7-3fb5-41ab-96c6-e69c8d004271 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:13:48.957870 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:13:49.026404 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-8778b0d7-3fb5-41ab-96c6-e69c8d004271 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8c6410322774d1f8cf8b3a0cadc19fb', 'request_id': 'req-8778b0d7-3fb5-41ab-96c6-e69c8d004271'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:13:49.031750 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-8778b0d7-3fb5-41ab-96c6-e69c8d004271 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8c6410322774d1f8cf8b3a0cadc19fb', 'request_id': 'req-8778b0d7-3fb5-41ab-96c6-e69c8d004271'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:13:49.086331 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-8778b0d7-3fb5-41ab-96c6-e69c8d004271 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8c6410322774d1f8cf8b3a0cadc19fb', 'request_id': 'req-8778b0d7-3fb5-41ab-96c6-e69c8d004271'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:13:49.097371 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-8778b0d7-3fb5-41ab-96c6-e69c8d004271 admin admin] The token cache is being invalidate because project b8c6410322774d1f8cf8b3a0cadc19fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 09:13:49.100846 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-8778b0d7-3fb5-41ab-96c6-e69c8d004271 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:13:49.100846 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-8778b0d7-3fb5-41ab-96c6-e69c8d004271 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:13:49.100846 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1302/2593] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 09:13:48 2022] DELETE /identity/v3/projects/b8c6410322774d1f8cf8b3a0cadc19fb => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:13:49.104440 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b7d04e-5240-4fc6-a49e-7e7a1233d70d admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:13:49.105850 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-98b7d04e-5240-4fc6-a49e-7e7a1233d70d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:13:49.118793 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b7d04e-5240-4fc6-a49e-7e7a1233d70d admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:13:49.173654 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909c2afe-0e56-4915-be64-6891021c2ae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:13:49.174269 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909c2afe-0e56-4915-be64-6891021c2ae2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:13:49.174545 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909c2afe-0e56-4915-be64-6891021c2ae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:13:49.174812 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909c2afe-0e56-4915-be64-6891021c2ae2 admin admin] PATH_INFO: `/v3/users/10d533f02e6a4f6184dfaf8525101d8b` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:13:49.175906 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909c2afe-0e56-4915-be64-6891021c2ae2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=10d533f02e6a4f6184dfaf8525101d8b)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:13:49.178349 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909c2afe-0e56-4915-be64-6891021c2ae2 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:13:49.368651 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:13:49.380422 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-909c2afe-0e56-4915-be64-6891021c2ae2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10d533f02e6a4f6184dfaf8525101d8b'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:13:49.393486 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-909c2afe-0e56-4915-be64-6891021c2ae2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10d533f02e6a4f6184dfaf8525101d8b'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:13:49.400682 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-909c2afe-0e56-4915-be64-6891021c2ae2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10d533f02e6a4f6184dfaf8525101d8b'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:13:49.402315 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-909c2afe-0e56-4915-be64-6891021c2ae2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10d533f02e6a4f6184dfaf8525101d8b'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:13:49.402315 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-909c2afe-0e56-4915-be64-6891021c2ae2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10d533f02e6a4f6184dfaf8525101d8b'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:13:49.469210 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1292/2594] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 09:13:49 2022] DELETE /identity/v3/users/10d533f02e6a4f6184dfaf8525101d8b => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:13:49.876268 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8778b0d7-3fb5-41ab-96c6-e69c8d004271 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:13:49.877868 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-8778b0d7-3fb5-41ab-96c6-e69c8d004271 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:13:49.891391 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8778b0d7-3fb5-41ab-96c6-e69c8d004271 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:13:49.948691 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e2ea84-68ab-4ff7-8e9b-237f2088dc42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:13:49.949340 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e2ea84-68ab-4ff7-8e9b-237f2088dc42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:13:49.949617 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e2ea84-68ab-4ff7-8e9b-237f2088dc42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:13:49.949884 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e2ea84-68ab-4ff7-8e9b-237f2088dc42 admin admin] PATH_INFO: `/v3/projects/91c7b18993904001b076559f9bcdb89f` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:13:49.950983 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e2ea84-68ab-4ff7-8e9b-237f2088dc42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91c7b18993904001b076559f9bcdb89f)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:13:49.952873 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e2ea84-68ab-4ff7-8e9b-237f2088dc42 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:13:50.052747 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-a2e2ea84-68ab-4ff7-8e9b-237f2088dc42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91c7b18993904001b076559f9bcdb89f', 'request_id': 'req-a2e2ea84-68ab-4ff7-8e9b-237f2088dc42'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:13:50.064981 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-a2e2ea84-68ab-4ff7-8e9b-237f2088dc42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91c7b18993904001b076559f9bcdb89f', 'request_id': 'req-a2e2ea84-68ab-4ff7-8e9b-237f2088dc42'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:13:50.126455 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-a2e2ea84-68ab-4ff7-8e9b-237f2088dc42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91c7b18993904001b076559f9bcdb89f', 'request_id': 'req-a2e2ea84-68ab-4ff7-8e9b-237f2088dc42'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:13:50.138860 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-a2e2ea84-68ab-4ff7-8e9b-237f2088dc42 admin admin] The token cache is being invalidate because project 91c7b18993904001b076559f9bcdb89f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 09:13:50.139212 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-a2e2ea84-68ab-4ff7-8e9b-237f2088dc42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:13:50.139835 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-a2e2ea84-68ab-4ff7-8e9b-237f2088dc42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:13:50.141653 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1303/2595] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 09:13:49 2022] DELETE /identity/v3/projects/91c7b18993904001b076559f9bcdb89f => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:13:50.148943 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41b0805-56f2-44f6-85c7-c194be995cfc None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:13:50.150290 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41b0805-56f2-44f6-85c7-c194be995cfc None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:13:50.150290 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41b0805-56f2-44f6-85c7-c194be995cfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:13:50.169339 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-d41b0805-56f2-44f6-85c7-c194be995cfc None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:13:50.173362 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-d41b0805-56f2-44f6-85c7-c194be995cfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:13:50.231285 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1293/2596] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 09:13:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:13:50.241538 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d69da82-902a-43c5-bcf4-f5a34d268a7e None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:13:50.243932 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d69da82-902a-43c5-bcf4-f5a34d268a7e None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:13:50.243932 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d69da82-902a-43c5-bcf4-f5a34d268a7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:13:50.262532 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-1d69da82-902a-43c5-bcf4-f5a34d268a7e None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:13:50.263768 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-1d69da82-902a-43c5-bcf4-f5a34d268a7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:13:50.272225 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1304/2597] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 09:13:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:13:50.276507 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41b0805-56f2-44f6-85c7-c194be995cfc None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:13:50.286404 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41b0805-56f2-44f6-85c7-c194be995cfc None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:13:50.291050 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e382271-a31e-4117-ab85-ecee457ed395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:13:50.295594 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e382271-a31e-4117-ab85-ecee457ed395 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:13:50.295594 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e382271-a31e-4117-ab85-ecee457ed395 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:13:50.295594 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e382271-a31e-4117-ab85-ecee457ed395 admin admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:13:50.295594 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e382271-a31e-4117-ab85-ecee457ed395 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:13:50.295594 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e382271-a31e-4117-ab85-ecee457ed395 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:13:50.295594 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e382271-a31e-4117-ab85-ecee457ed395 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:13:50.311530 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1294/2598] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Aug 10 09:13:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:13:50.312949 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d69da82-902a-43c5-bcf4-f5a34d268a7e None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:13:50.323141 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d69da82-902a-43c5-bcf4-f5a34d268a7e None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:13:50.327844 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c5e03d-9d92-4194-82c2-72cb2a409110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:13:50.328442 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c5e03d-9d92-4194-82c2-72cb2a409110 admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:13:50.328715 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c5e03d-9d92-4194-82c2-72cb2a409110 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:13:50.329014 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c5e03d-9d92-4194-82c2-72cb2a409110 admin admin] PATH_INFO: `/v3/projects` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:13:50.329667 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c5e03d-9d92-4194-82c2-72cb2a409110 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:13:50.331606 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c5e03d-9d92-4194-82c2-72cb2a409110 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:13:50.431461 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1305/2599] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 09:13:50 2022] POST /identity/v3/projects => generated 379 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 09:13:50.435815 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e382271-a31e-4117-ab85-ecee457ed395 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:13:50.447754 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e382271-a31e-4117-ab85-ecee457ed395 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:13:50.503889 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446fe5b4-9632-4c06-90bb-a0f4245b5870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:13:50.504486 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446fe5b4-9632-4c06-90bb-a0f4245b5870 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:13:50.504760 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446fe5b4-9632-4c06-90bb-a0f4245b5870 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:13:50.505057 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446fe5b4-9632-4c06-90bb-a0f4245b5870 admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:13:50.505671 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446fe5b4-9632-4c06-90bb-a0f4245b5870 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:13:50.508391 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446fe5b4-9632-4c06-90bb-a0f4245b5870 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:13:50.612723 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1295/2600] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 09:13:50 2022] POST /identity/v3/users => generated 290 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 09:13:50.617567 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c5e03d-9d92-4194-82c2-72cb2a409110 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:13:50.623343 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:13:50.628853 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c5e03d-9d92-4194-82c2-72cb2a409110 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:13:50.634663 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e08d317-d10a-445a-85fd-675f758dc475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:13:50.635289 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e08d317-d10a-445a-85fd-675f758dc475 admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:13:50.635564 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e08d317-d10a-445a-85fd-675f758dc475 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:13:50.635830 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e08d317-d10a-445a-85fd-675f758dc475 admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:13:50.636470 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e08d317-d10a-445a-85fd-675f758dc475 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:13:50.638115 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e08d317-d10a-445a-85fd-675f758dc475 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:13:50.648522 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1306/2601] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 09:13:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 09:13:50.652746 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446fe5b4-9632-4c06-90bb-a0f4245b5870 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:13:50.663613 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446fe5b4-9632-4c06-90bb-a0f4245b5870 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:13:50.668517 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732d3447-0e1e-46bc-894a-696b7e8c924e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:13:50.669142 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732d3447-0e1e-46bc-894a-696b7e8c924e admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:13:50.669424 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732d3447-0e1e-46bc-894a-696b7e8c924e admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:13:50.669695 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732d3447-0e1e-46bc-894a-696b7e8c924e admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:13:50.670331 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732d3447-0e1e-46bc-894a-696b7e8c924e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:13:50.671957 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732d3447-0e1e-46bc-894a-696b7e8c924e admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:13:50.682784 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1296/2602] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 09:13:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 09:13:50.686998 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e08d317-d10a-445a-85fd-675f758dc475 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:13:50.697537 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e08d317-d10a-445a-85fd-675f758dc475 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:13:50.702156 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b808cb-96e9-43f0-9e64-ec65779a67a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:13:50.702800 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b808cb-96e9-43f0-9e64-ec65779a67a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:13:50.703081 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b808cb-96e9-43f0-9e64-ec65779a67a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:13:50.703379 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b808cb-96e9-43f0-9e64-ec65779a67a7 admin admin] PATH_INFO: `/v3/projects/0a7f8ecfa8944b2f9c7b96f459bcf733/users/869760704faf4aa28e374ef988b52257/roles/44b64db484f24a46bf0b7ced46b519bf` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:13:50.719473 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b808cb-96e9-43f0-9e64-ec65779a67a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a7f8ecfa8944b2f9c7b96f459bcf733, user_id=869760704faf4aa28e374ef988b52257, role_id=44b64db484f24a46bf0b7ced46b519bf)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:13:50.722557 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b808cb-96e9-43f0-9e64-ec65779a67a7 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:13:50.794540 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1307/2603] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 09:13:50 2022] PUT /identity/v3/projects/0a7f8ecfa8944b2f9c7b96f459bcf733/users/869760704faf4aa28e374ef988b52257/roles/44b64db484f24a46bf0b7ced46b519bf => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:13:50.806998 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6f602b-5e29-48c9-aa9b-247344276c79 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:13:50.807284 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6f602b-5e29-48c9-aa9b-247344276c79 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:13:50.807559 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6f602b-5e29-48c9-aa9b-247344276c79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:13:50.836182 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-3f6f602b-5e29-48c9-aa9b-247344276c79 None None] MFA Rules not processed for user `869760704faf4aa28e374ef988b52257`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:13:50.837438 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-3f6f602b-5e29-48c9-aa9b-247344276c79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:13:50.841524 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:13:50.903182 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1297/2604] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 09:13:50 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:13:50.907534 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b808cb-96e9-43f0-9e64-ec65779a67a7 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:13:50.969303 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b808cb-96e9-43f0-9e64-ec65779a67a7 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:13:50.973897 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c53cf46-975a-4015-ae8e-ad1a9320c4ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:13:50.974482 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c53cf46-975a-4015-ae8e-ad1a9320c4ca admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:13:50.975247 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c53cf46-975a-4015-ae8e-ad1a9320c4ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:13:50.975519 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c53cf46-975a-4015-ae8e-ad1a9320c4ca admin admin] PATH_INFO: `/v3/projects` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:13:50.976130 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c53cf46-975a-4015-ae8e-ad1a9320c4ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:13:50.978054 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c53cf46-975a-4015-ae8e-ad1a9320c4ca admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:13:51.052628 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1308/2605] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 09:13:50 2022] POST /identity/v3/projects => generated 377 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 09:13:51.057408 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6f602b-5e29-48c9-aa9b-247344276c79 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:13:51.068104 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6f602b-5e29-48c9-aa9b-247344276c79 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:13:51.128528 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2c1a88-3d89-46c6-a288-2a43b2bf32ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:13:51.129178 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2c1a88-3d89-46c6-a288-2a43b2bf32ed admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:13:51.129457 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2c1a88-3d89-46c6-a288-2a43b2bf32ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:13:51.129726 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2c1a88-3d89-46c6-a288-2a43b2bf32ed admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:13:51.130350 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2c1a88-3d89-46c6-a288-2a43b2bf32ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:13:51.133016 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2c1a88-3d89-46c6-a288-2a43b2bf32ed admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:13:51.211847 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1298/2606] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 09:13:51 2022] POST /identity/v3/users => generated 295 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 09:13:51.215803 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c53cf46-975a-4015-ae8e-ad1a9320c4ca admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:13:51.226732 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c53cf46-975a-4015-ae8e-ad1a9320c4ca admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:13:51.231808 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bef7e8-9014-483f-a29c-3b9248ed6a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:13:51.232435 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bef7e8-9014-483f-a29c-3b9248ed6a73 admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:13:51.232708 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bef7e8-9014-483f-a29c-3b9248ed6a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:13:51.233015 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bef7e8-9014-483f-a29c-3b9248ed6a73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:13:51.233726 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bef7e8-9014-483f-a29c-3b9248ed6a73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:13:51.235356 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bef7e8-9014-483f-a29c-3b9248ed6a73 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:13:51.247874 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1309/2607] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 09:13:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 09:13:51.252091 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2c1a88-3d89-46c6-a288-2a43b2bf32ed admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:13:51.262028 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2c1a88-3d89-46c6-a288-2a43b2bf32ed admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:13:51.266840 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ff32a8-67c9-4174-a655-ac3fab84a987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:13:51.267472 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ff32a8-67c9-4174-a655-ac3fab84a987 admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:13:51.267746 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ff32a8-67c9-4174-a655-ac3fab84a987 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:13:51.268009 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ff32a8-67c9-4174-a655-ac3fab84a987 admin admin] PATH_INFO: `/v3/projects/c31b2325063d4330bf28c6a42cf05d32/users/4de2c4802af64d53b9b6cab5da46fb8e/roles/ec3751692d9c4ecf81b4a71c936c46be` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:13:51.282625 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ff32a8-67c9-4174-a655-ac3fab84a987 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c31b2325063d4330bf28c6a42cf05d32, user_id=4de2c4802af64d53b9b6cab5da46fb8e, role_id=ec3751692d9c4ecf81b4a71c936c46be)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:13:51.285320 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ff32a8-67c9-4174-a655-ac3fab84a987 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:13:51.355496 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1299/2608] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 09:13:51 2022] PUT /identity/v3/projects/c31b2325063d4330bf28c6a42cf05d32/users/4de2c4802af64d53b9b6cab5da46fb8e/roles/ec3751692d9c4ecf81b4a71c936c46be => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:13:51.359741 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bef7e8-9014-483f-a29c-3b9248ed6a73 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:13:51.371937 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bef7e8-9014-483f-a29c-3b9248ed6a73 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:13:51.436543 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f2a153-892c-4400-886f-98430bf7b6e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:13:51.437188 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f2a153-892c-4400-886f-98430bf7b6e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:13:51.437465 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f2a153-892c-4400-886f-98430bf7b6e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:13:51.437733 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f2a153-892c-4400-886f-98430bf7b6e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:13:51.439345 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f2a153-892c-4400-886f-98430bf7b6e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:13:51.440946 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f2a153-892c-4400-886f-98430bf7b6e9 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:13:51.450765 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1310/2609] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 09:13:51 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 09:13:51.454971 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ff32a8-67c9-4174-a655-ac3fab84a987 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:13:51.465373 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ff32a8-67c9-4174-a655-ac3fab84a987 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:13:51.469911 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b547bb79-48f0-4578-8b3d-61794189a2d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:13:51.470580 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b547bb79-48f0-4578-8b3d-61794189a2d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:13:51.470860 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b547bb79-48f0-4578-8b3d-61794189a2d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:13:51.471131 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b547bb79-48f0-4578-8b3d-61794189a2d2 admin admin] PATH_INFO: `/v3/projects/c31b2325063d4330bf28c6a42cf05d32/users/4de2c4802af64d53b9b6cab5da46fb8e/roles/ec3751692d9c4ecf81b4a71c936c46be` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:13:51.473061 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b547bb79-48f0-4578-8b3d-61794189a2d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c31b2325063d4330bf28c6a42cf05d32, user_id=4de2c4802af64d53b9b6cab5da46fb8e, role_id=ec3751692d9c4ecf81b4a71c936c46be)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:13:51.476340 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b547bb79-48f0-4578-8b3d-61794189a2d2 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:13:51.486396 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1300/2610] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 09:13:51 2022] PUT /identity/v3/projects/c31b2325063d4330bf28c6a42cf05d32/users/4de2c4802af64d53b9b6cab5da46fb8e/roles/ec3751692d9c4ecf81b4a71c936c46be => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:13:51.497342 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9a0162-30b7-4cd2-90b9-010a81715c7b None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:13:51.497634 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9a0162-30b7-4cd2-90b9-010a81715c7b None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:13:51.497911 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9a0162-30b7-4cd2-90b9-010a81715c7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:13:51.525503 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-2a9a0162-30b7-4cd2-90b9-010a81715c7b None None] MFA Rules not processed for user `4de2c4802af64d53b9b6cab5da46fb8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:13:51.526706 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-2a9a0162-30b7-4cd2-90b9-010a81715c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:13:51.553554 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:13:51.593604 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1311/2611] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 09:13:51 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:13:51.605451 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b547bb79-48f0-4578-8b3d-61794189a2d2 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:13:51.607180 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-b547bb79-48f0-4578-8b3d-61794189a2d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:13:51.687435 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:13:51.692817 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b547bb79-48f0-4578-8b3d-61794189a2d2 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:13:51.703276 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be58c127-62a5-4ad1-b29b-fef4de14d434 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ba74e28ced4cbba857f54222dfbe41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:13:51.703276 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be58c127-62a5-4ad1-b29b-fef4de14d434 service cinder] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:13:51.703276 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be58c127-62a5-4ad1-b29b-fef4de14d434 service cinder] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:13:51.703276 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be58c127-62a5-4ad1-b29b-fef4de14d434 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:13:51.707313 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be58c127-62a5-4ad1-b29b-fef4de14d434 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:13:51.708724 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be58c127-62a5-4ad1-b29b-fef4de14d434 service cinder] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:13:51.719437 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1301/2612] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:13:51 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:13:52.242118 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9a0162-30b7-4cd2-90b9-010a81715c7b None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:13:52.305304 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9a0162-30b7-4cd2-90b9-010a81715c7b None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:13:52.309730 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a72e695-562f-4560-8014-666e8d820d86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:13:52.310348 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a72e695-562f-4560-8014-666e8d820d86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:13:52.310623 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a72e695-562f-4560-8014-666e8d820d86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:13:52.310895 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a72e695-562f-4560-8014-666e8d820d86 admin admin] PATH_INFO: `/v3/users/869760704faf4aa28e374ef988b52257` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:13:52.311987 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a72e695-562f-4560-8014-666e8d820d86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=869760704faf4aa28e374ef988b52257)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:13:52.313938 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a72e695-562f-4560-8014-666e8d820d86 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:13:52.500410 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-2a72e695-562f-4560-8014-666e8d820d86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '869760704faf4aa28e374ef988b52257'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:13:52.513119 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-2a72e695-562f-4560-8014-666e8d820d86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '869760704faf4aa28e374ef988b52257'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:13:52.520222 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-2a72e695-562f-4560-8014-666e8d820d86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '869760704faf4aa28e374ef988b52257'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:13:52.577414 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-2a72e695-562f-4560-8014-666e8d820d86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '869760704faf4aa28e374ef988b52257'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:13:52.578192 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-2a72e695-562f-4560-8014-666e8d820d86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '869760704faf4aa28e374ef988b52257'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:13:52.584451 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1312/2613] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 09:13:52 2022] DELETE /identity/v3/users/869760704faf4aa28e374ef988b52257 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:13:52.588909 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b99a47-258e-49cf-a5c7-77246ecab459 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:13:52.589387 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b99a47-258e-49cf-a5c7-77246ecab459 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:13:52.589585 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b99a47-258e-49cf-a5c7-77246ecab459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:13:52.608722 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-61b99a47-258e-49cf-a5c7-77246ecab459 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:13:52.609945 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-61b99a47-258e-49cf-a5c7-77246ecab459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:13:52.669469 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1302/2614] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 09:13:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:13:52.679512 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a72e695-562f-4560-8014-666e8d820d86 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:13:52.680978 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-2a72e695-562f-4560-8014-666e8d820d86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:13:52.726607 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:13:52.738449 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a72e695-562f-4560-8014-666e8d820d86 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:13:52.743243 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb7229a-c39f-46aa-83ce-2e6b86e828a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:13:52.743853 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb7229a-c39f-46aa-83ce-2e6b86e828a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:13:52.744131 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb7229a-c39f-46aa-83ce-2e6b86e828a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:13:52.744400 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb7229a-c39f-46aa-83ce-2e6b86e828a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:13:52.753707 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb7229a-c39f-46aa-83ce-2e6b86e828a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:13:52.755603 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb7229a-c39f-46aa-83ce-2e6b86e828a5 service neutron] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:13:52.766832 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1313/2615] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:13:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:13:53.179330 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b99a47-258e-49cf-a5c7-77246ecab459 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:13:53.183053 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-61b99a47-258e-49cf-a5c7-77246ecab459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:13:53.197235 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b99a47-258e-49cf-a5c7-77246ecab459 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:13:53.201996 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2c775a-0c80-4a35-adbc-7cb5b75aecb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:13:53.202787 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2c775a-0c80-4a35-adbc-7cb5b75aecb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:13:53.202923 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2c775a-0c80-4a35-adbc-7cb5b75aecb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:13:53.203177 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2c775a-0c80-4a35-adbc-7cb5b75aecb2 admin admin] PATH_INFO: `/v3/projects/0a7f8ecfa8944b2f9c7b96f459bcf733` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:13:53.204328 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2c775a-0c80-4a35-adbc-7cb5b75aecb2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a7f8ecfa8944b2f9c7b96f459bcf733)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:13:53.206410 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2c775a-0c80-4a35-adbc-7cb5b75aecb2 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:13:53.322571 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-ce2c775a-0c80-4a35-adbc-7cb5b75aecb2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a7f8ecfa8944b2f9c7b96f459bcf733', 'request_id': 'req-ce2c775a-0c80-4a35-adbc-7cb5b75aecb2'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:13:53.382916 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-ce2c775a-0c80-4a35-adbc-7cb5b75aecb2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a7f8ecfa8944b2f9c7b96f459bcf733', 'request_id': 'req-ce2c775a-0c80-4a35-adbc-7cb5b75aecb2'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:13:53.389061 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-ce2c775a-0c80-4a35-adbc-7cb5b75aecb2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a7f8ecfa8944b2f9c7b96f459bcf733', 'request_id': 'req-ce2c775a-0c80-4a35-adbc-7cb5b75aecb2'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:13:53.400436 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-ce2c775a-0c80-4a35-adbc-7cb5b75aecb2 admin admin] The token cache is being invalidate because project 0a7f8ecfa8944b2f9c7b96f459bcf733 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 09:13:53.400728 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-ce2c775a-0c80-4a35-adbc-7cb5b75aecb2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:13:53.401406 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-ce2c775a-0c80-4a35-adbc-7cb5b75aecb2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:13:53.402903 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1303/2616] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 09:13:53 2022] DELETE /identity/v3/projects/0a7f8ecfa8944b2f9c7b96f459bcf733 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:13:53.406854 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb7229a-c39f-46aa-83ce-2e6b86e828a5 service neutron] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:13:53.416837 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb7229a-c39f-46aa-83ce-2e6b86e828a5 service neutron] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:13:53.472387 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee84844-eb57-4b96-81d5-0309a913b12b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:13:53.473032 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee84844-eb57-4b96-81d5-0309a913b12b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:13:53.473311 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee84844-eb57-4b96-81d5-0309a913b12b admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:13:53.473582 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee84844-eb57-4b96-81d5-0309a913b12b admin admin] PATH_INFO: `/v3/users/4de2c4802af64d53b9b6cab5da46fb8e` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:13:53.474792 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee84844-eb57-4b96-81d5-0309a913b12b admin admin] RBAC: Authorizing `identity:delete_user(user_id=4de2c4802af64d53b9b6cab5da46fb8e)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:13:53.476712 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee84844-eb57-4b96-81d5-0309a913b12b admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:13:53.625373 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-2ee84844-eb57-4b96-81d5-0309a913b12b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4de2c4802af64d53b9b6cab5da46fb8e'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:13:53.637952 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-2ee84844-eb57-4b96-81d5-0309a913b12b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4de2c4802af64d53b9b6cab5da46fb8e'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:13:53.645044 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-2ee84844-eb57-4b96-81d5-0309a913b12b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4de2c4802af64d53b9b6cab5da46fb8e'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:13:53.795030 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-2ee84844-eb57-4b96-81d5-0309a913b12b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4de2c4802af64d53b9b6cab5da46fb8e'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:13:53.795653 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-2ee84844-eb57-4b96-81d5-0309a913b12b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4de2c4802af64d53b9b6cab5da46fb8e'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:13:53.801933 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1314/2617] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 09:13:53 2022] DELETE /identity/v3/users/4de2c4802af64d53b9b6cab5da46fb8e => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:13:54.213862 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2c775a-0c80-4a35-adbc-7cb5b75aecb2 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:13:54.215286 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-ce2c775a-0c80-4a35-adbc-7cb5b75aecb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:13:54.229485 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2c775a-0c80-4a35-adbc-7cb5b75aecb2 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:13:54.302373 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d557ab9-c03f-4014-9b2f-1c06fec10471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:13:54.304106 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d557ab9-c03f-4014-9b2f-1c06fec10471 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:13:54.304386 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d557ab9-c03f-4014-9b2f-1c06fec10471 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:13:54.304654 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d557ab9-c03f-4014-9b2f-1c06fec10471 admin admin] PATH_INFO: `/v3/projects/c31b2325063d4330bf28c6a42cf05d32` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:13:54.305818 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d557ab9-c03f-4014-9b2f-1c06fec10471 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c31b2325063d4330bf28c6a42cf05d32)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:13:54.307772 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d557ab9-c03f-4014-9b2f-1c06fec10471 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:13:54.324584 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:13:54.441424 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-5d557ab9-c03f-4014-9b2f-1c06fec10471 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c31b2325063d4330bf28c6a42cf05d32', 'request_id': 'req-5d557ab9-c03f-4014-9b2f-1c06fec10471'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:13:54.510716 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-5d557ab9-c03f-4014-9b2f-1c06fec10471 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c31b2325063d4330bf28c6a42cf05d32', 'request_id': 'req-5d557ab9-c03f-4014-9b2f-1c06fec10471'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:13:54.516900 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-5d557ab9-c03f-4014-9b2f-1c06fec10471 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c31b2325063d4330bf28c6a42cf05d32', 'request_id': 'req-5d557ab9-c03f-4014-9b2f-1c06fec10471'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:13:54.528713 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-5d557ab9-c03f-4014-9b2f-1c06fec10471 admin admin] The token cache is being invalidate because project c31b2325063d4330bf28c6a42cf05d32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 09:13:54.528965 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-5d557ab9-c03f-4014-9b2f-1c06fec10471 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:13:54.529679 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-5d557ab9-c03f-4014-9b2f-1c06fec10471 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:13:54.531516 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1304/2618] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 09:13:54 2022] DELETE /identity/v3/projects/c31b2325063d4330bf28c6a42cf05d32 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:13:54.536000 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c843cca9-0aec-495f-b2e9-5b4393a98266 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:13:54.536495 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c843cca9-0aec-495f-b2e9-5b4393a98266 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:13:54.536894 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c843cca9-0aec-495f-b2e9-5b4393a98266 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:13:54.557506 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-c843cca9-0aec-495f-b2e9-5b4393a98266 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:13:54.558720 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-c843cca9-0aec-495f-b2e9-5b4393a98266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:13:54.619838 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1315/2619] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 09:13:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 09:13:54.627699 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d3827f-257c-4371-af7b-f9443d25af9e None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:13:54.627979 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d3827f-257c-4371-af7b-f9443d25af9e None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:13:54.628246 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d3827f-257c-4371-af7b-f9443d25af9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:13:54.650005 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-99d3827f-257c-4371-af7b-f9443d25af9e None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:13:54.651189 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-99d3827f-257c-4371-af7b-f9443d25af9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:13:54.658907 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1305/2620] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 09:13:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:13:54.663080 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c843cca9-0aec-495f-b2e9-5b4393a98266 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:13:54.669103 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:13:54.672828 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c843cca9-0aec-495f-b2e9-5b4393a98266 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:13:54.677673 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22870870-f7cd-4d36-a8ad-7249b45f6f77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:13:54.678331 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22870870-f7cd-4d36-a8ad-7249b45f6f77 admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:13:54.678616 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22870870-f7cd-4d36-a8ad-7249b45f6f77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:13:54.678908 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22870870-f7cd-4d36-a8ad-7249b45f6f77 admin admin] PATH_INFO: `/v3/domains` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:13:54.679781 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22870870-f7cd-4d36-a8ad-7249b45f6f77 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:13:54.680089 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22870870-f7cd-4d36-a8ad-7249b45f6f77 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:13:54.681736 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22870870-f7cd-4d36-a8ad-7249b45f6f77 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:13:54.693045 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1316/2621] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Aug 10 09:13:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:13:54.697944 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d3827f-257c-4371-af7b-f9443d25af9e None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:13:54.707221 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d3827f-257c-4371-af7b-f9443d25af9e None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:13:54.712555 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048a90ec-fdd1-46d9-910d-e4a8b5ae34b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:13:54.713195 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048a90ec-fdd1-46d9-910d-e4a8b5ae34b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:13:54.713472 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048a90ec-fdd1-46d9-910d-e4a8b5ae34b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:13:54.713742 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048a90ec-fdd1-46d9-910d-e4a8b5ae34b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:13:54.714374 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048a90ec-fdd1-46d9-910d-e4a8b5ae34b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:13:54.716270 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048a90ec-fdd1-46d9-910d-e4a8b5ae34b2 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:13:54.790703 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1306/2622] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 09:13:54 2022] POST /identity/v3/projects => generated 373 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 09:13:54.794564 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22870870-f7cd-4d36-a8ad-7249b45f6f77 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:13:54.804638 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22870870-f7cd-4d36-a8ad-7249b45f6f77 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:13:54.858530 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbe8f3d-2152-4281-9f61-af3e2d8f6a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:13:54.859123 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbe8f3d-2152-4281-9f61-af3e2d8f6a03 admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:13:54.859403 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbe8f3d-2152-4281-9f61-af3e2d8f6a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:13:54.859677 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbe8f3d-2152-4281-9f61-af3e2d8f6a03 admin admin] PATH_INFO: `/v3/users` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:13:54.860300 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbe8f3d-2152-4281-9f61-af3e2d8f6a03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:13:54.862252 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbe8f3d-2152-4281-9f61-af3e2d8f6a03 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:13:54.952730 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1317/2623] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 09:13:54 2022] POST /identity/v3/users => generated 287 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 09:13:54.957261 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048a90ec-fdd1-46d9-910d-e4a8b5ae34b2 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:13:54.968362 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048a90ec-fdd1-46d9-910d-e4a8b5ae34b2 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:13:54.978084 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c04426-ecd7-4c3c-b16d-f989bab67e4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:13:54.978699 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c04426-ecd7-4c3c-b16d-f989bab67e4d admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:13:54.978979 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c04426-ecd7-4c3c-b16d-f989bab67e4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:13:54.979252 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c04426-ecd7-4c3c-b16d-f989bab67e4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:13:54.979916 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c04426-ecd7-4c3c-b16d-f989bab67e4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:13:54.981556 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c04426-ecd7-4c3c-b16d-f989bab67e4d admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:13:54.997422 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1307/2624] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 09:13:54 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 09:13:54.998602 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbe8f3d-2152-4281-9f61-af3e2d8f6a03 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:13:55.009392 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbe8f3d-2152-4281-9f61-af3e2d8f6a03 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:13:55.014004 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64023372-abef-469c-978b-169aa844fe54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:13:55.014604 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64023372-abef-469c-978b-169aa844fe54 admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:13:55.014882 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64023372-abef-469c-978b-169aa844fe54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:13:55.015154 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64023372-abef-469c-978b-169aa844fe54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:13:55.016064 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64023372-abef-469c-978b-169aa844fe54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:13:55.017715 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64023372-abef-469c-978b-169aa844fe54 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:13:55.028812 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1318/2625] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 09:13:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 09:13:55.032786 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c04426-ecd7-4c3c-b16d-f989bab67e4d admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:13:55.042924 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c04426-ecd7-4c3c-b16d-f989bab67e4d admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:13:55.047513 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e3d9bc-6383-4d49-8445-d64628118fef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:13:55.048158 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e3d9bc-6383-4d49-8445-d64628118fef admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:13:55.048438 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e3d9bc-6383-4d49-8445-d64628118fef admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:13:55.048710 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e3d9bc-6383-4d49-8445-d64628118fef admin admin] PATH_INFO: `/v3/projects/6b2c0dc9dfab4bad8e6dd81cebf6a16f/users/84219759abdb4509aef31cbc4dcbdb08/roles/44b64db484f24a46bf0b7ced46b519bf` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:13:55.063543 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e3d9bc-6383-4d49-8445-d64628118fef admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b2c0dc9dfab4bad8e6dd81cebf6a16f, user_id=84219759abdb4509aef31cbc4dcbdb08, role_id=44b64db484f24a46bf0b7ced46b519bf)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:13:55.066292 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e3d9bc-6383-4d49-8445-d64628118fef admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:13:55.139713 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1308/2626] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 09:13:55 2022] PUT /identity/v3/projects/6b2c0dc9dfab4bad8e6dd81cebf6a16f/users/84219759abdb4509aef31cbc4dcbdb08/roles/44b64db484f24a46bf0b7ced46b519bf => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:13:55.145356 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed34887c-7ef7-467a-b8a4-b09b4b7f6980 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:13:55.145641 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed34887c-7ef7-467a-b8a4-b09b4b7f6980 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:13:55.145909 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed34887c-7ef7-467a-b8a4-b09b4b7f6980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:13:55.167221 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-ed34887c-7ef7-467a-b8a4-b09b4b7f6980 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:13:55.168412 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-ed34887c-7ef7-467a-b8a4-b09b4b7f6980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:13:55.233335 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1319/2627] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 09:13:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:13:55.248151 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e3d9bc-6383-4d49-8445-d64628118fef admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:13:55.250171 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-89e3d9bc-6383-4d49-8445-d64628118fef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:13:55.310975 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e3d9bc-6383-4d49-8445-d64628118fef admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:13:55.314623 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e433d4-19b3-4061-b559-eaf143a48457 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:13:55.315232 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e433d4-19b3-4061-b559-eaf143a48457 service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:13:55.315513 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e433d4-19b3-4061-b559-eaf143a48457 service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:13:55.315789 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e433d4-19b3-4061-b559-eaf143a48457 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:13:55.326884 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e433d4-19b3-4061-b559-eaf143a48457 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:13:55.328318 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e433d4-19b3-4061-b559-eaf143a48457 service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:13:55.338380 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1309/2628] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:13:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:13:56.597359 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1bdbe5-d9c0-4962-b6bf-7d344707fa0c None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:13:56.597911 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1bdbe5-d9c0-4962-b6bf-7d344707fa0c None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:13:56.598049 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1bdbe5-d9c0-4962-b6bf-7d344707fa0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:13:56.629371 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-9c1bdbe5-d9c0-4962-b6bf-7d344707fa0c None None] MFA Rules not processed for user `84219759abdb4509aef31cbc4dcbdb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:13:56.630608 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-9c1bdbe5-d9c0-4962-b6bf-7d344707fa0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:13:56.696352 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:13:56.701488 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1320/2629] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 09:13:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:13:56.704651 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e433d4-19b3-4061-b559-eaf143a48457 service neutron] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:13:56.718802 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e433d4-19b3-4061-b559-eaf143a48457 service neutron] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:13:56.723457 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2f1dc9-4631-44c9-ac75-287f860acde2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:13:56.724206 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2f1dc9-4631-44c9-ac75-287f860acde2 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:13:56.724206 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2f1dc9-4631-44c9-ac75-287f860acde2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:13:56.724617 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2f1dc9-4631-44c9-ac75-287f860acde2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:13:56.725203 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2f1dc9-4631-44c9-ac75-287f860acde2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:13:56.727911 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2f1dc9-4631-44c9-ac75-287f860acde2 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:13:56.790367 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1310/2630] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 09:13:56 2022] POST /identity/v3/projects => generated 371 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 09:13:56.794414 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1bdbe5-d9c0-4962-b6bf-7d344707fa0c None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:13:56.805274 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1bdbe5-d9c0-4962-b6bf-7d344707fa0c None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:13:56.860676 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0e8fcf-dfb2-40b4-8db0-a8250060dc8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:13:56.861311 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0e8fcf-dfb2-40b4-8db0-a8250060dc8a admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:13:56.861590 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0e8fcf-dfb2-40b4-8db0-a8250060dc8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:13:56.861860 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0e8fcf-dfb2-40b4-8db0-a8250060dc8a admin admin] PATH_INFO: `/v3/users` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:13:56.862480 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0e8fcf-dfb2-40b4-8db0-a8250060dc8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:13:56.864382 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0e8fcf-dfb2-40b4-8db0-a8250060dc8a admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:13:56.926211 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1321/2631] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 09:13:56 2022] POST /identity/v3/users => generated 292 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 09:13:56.930262 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2f1dc9-4631-44c9-ac75-287f860acde2 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:13:56.940677 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2f1dc9-4631-44c9-ac75-287f860acde2 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:13:56.945981 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7962a1c4-be21-4c7f-a252-6524c2003fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:13:56.946569 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7962a1c4-be21-4c7f-a252-6524c2003fce admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:13:56.946846 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7962a1c4-be21-4c7f-a252-6524c2003fce admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:13:56.947113 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7962a1c4-be21-4c7f-a252-6524c2003fce admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:13:56.947782 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7962a1c4-be21-4c7f-a252-6524c2003fce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:13:56.949674 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7962a1c4-be21-4c7f-a252-6524c2003fce admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:13:57.085944 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1311/2632] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 09:13:56 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 09:13:57.090423 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0e8fcf-dfb2-40b4-8db0-a8250060dc8a admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:13:57.101707 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0e8fcf-dfb2-40b4-8db0-a8250060dc8a admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:13:57.106357 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bcbe15-70a6-444c-bdd2-e552a3c711c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:13:57.107000 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bcbe15-70a6-444c-bdd2-e552a3c711c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:13:57.107278 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bcbe15-70a6-444c-bdd2-e552a3c711c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:13:57.107545 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bcbe15-70a6-444c-bdd2-e552a3c711c4 admin admin] PATH_INFO: `/v3/projects/90ec6a9ff8374dc3913265622034ddf8/users/1a655a3e5aef453a86cf8e8ff39f423b/roles/ec3751692d9c4ecf81b4a71c936c46be` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:13:57.122676 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bcbe15-70a6-444c-bdd2-e552a3c711c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90ec6a9ff8374dc3913265622034ddf8, user_id=1a655a3e5aef453a86cf8e8ff39f423b, role_id=ec3751692d9c4ecf81b4a71c936c46be)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:13:57.125390 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bcbe15-70a6-444c-bdd2-e552a3c711c4 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:13:57.212381 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1322/2633] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 09:13:57 2022] PUT /identity/v3/projects/90ec6a9ff8374dc3913265622034ddf8/users/1a655a3e5aef453a86cf8e8ff39f423b/roles/ec3751692d9c4ecf81b4a71c936c46be => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:13:57.216440 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7962a1c4-be21-4c7f-a252-6524c2003fce admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:13:57.227316 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7962a1c4-be21-4c7f-a252-6524c2003fce admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:13:57.232066 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:13:57.285787 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c9eae9-d87d-4484-89aa-60a1e5448d60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:13:57.286394 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c9eae9-d87d-4484-89aa-60a1e5448d60 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:13:57.286675 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c9eae9-d87d-4484-89aa-60a1e5448d60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:13:57.286954 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c9eae9-d87d-4484-89aa-60a1e5448d60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:13:57.287621 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c9eae9-d87d-4484-89aa-60a1e5448d60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:13:57.289840 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c9eae9-d87d-4484-89aa-60a1e5448d60 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:13:57.299937 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1312/2634] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 09:13:57 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 09:13:57.304048 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bcbe15-70a6-444c-bdd2-e552a3c711c4 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:13:57.314377 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bcbe15-70a6-444c-bdd2-e552a3c711c4 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:13:57.319066 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21dcc0e-7018-4f7a-999e-2414865e4478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:13:57.319758 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21dcc0e-7018-4f7a-999e-2414865e4478 admin admin] REQUEST_METHOD: `PUT` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:13:57.320047 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21dcc0e-7018-4f7a-999e-2414865e4478 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:13:57.320320 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21dcc0e-7018-4f7a-999e-2414865e4478 admin admin] PATH_INFO: `/v3/projects/90ec6a9ff8374dc3913265622034ddf8/users/1a655a3e5aef453a86cf8e8ff39f423b/roles/ec3751692d9c4ecf81b4a71c936c46be` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:13:57.322320 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21dcc0e-7018-4f7a-999e-2414865e4478 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90ec6a9ff8374dc3913265622034ddf8, user_id=1a655a3e5aef453a86cf8e8ff39f423b, role_id=ec3751692d9c4ecf81b4a71c936c46be)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:13:57.325025 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21dcc0e-7018-4f7a-999e-2414865e4478 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:13:57.335488 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1323/2635] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 09:13:57 2022] PUT /identity/v3/projects/90ec6a9ff8374dc3913265622034ddf8/users/1a655a3e5aef453a86cf8e8ff39f423b/roles/ec3751692d9c4ecf81b4a71c936c46be => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:13:58.424539 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21acc972-2787-4279-84ff-bd5bfaf1f5ea None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:13:58.425068 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21acc972-2787-4279-84ff-bd5bfaf1f5ea None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:13:58.425201 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21acc972-2787-4279-84ff-bd5bfaf1f5ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:13:58.456573 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-21acc972-2787-4279-84ff-bd5bfaf1f5ea None None] MFA Rules not processed for user `1a655a3e5aef453a86cf8e8ff39f423b`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:13:58.457869 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-21acc972-2787-4279-84ff-bd5bfaf1f5ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:13:58.532882 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1313/2636] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 09:13:58 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:13:58.560688 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21dcc0e-7018-4f7a-999e-2414865e4478 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:13:58.562425 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-c21dcc0e-7018-4f7a-999e-2414865e4478 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:13:58.618453 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21dcc0e-7018-4f7a-999e-2414865e4478 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:13:58.621980 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b15aa96-ae4d-4174-bfbc-8bfbfdedbe74 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ba74e28ced4cbba857f54222dfbe41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:13:58.622577 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b15aa96-ae4d-4174-bfbc-8bfbfdedbe74 service cinder] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:13:58.622858 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b15aa96-ae4d-4174-bfbc-8bfbfdedbe74 service cinder] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:13:58.623132 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b15aa96-ae4d-4174-bfbc-8bfbfdedbe74 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:13:58.678325 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b15aa96-ae4d-4174-bfbc-8bfbfdedbe74 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:13:58.680106 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b15aa96-ae4d-4174-bfbc-8bfbfdedbe74 service cinder] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:13:58.688844 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1324/2637] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:13:58 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:14:08.529364 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=59 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:14:08.684829 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=92,keystone:INSERT=5 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:14:47.516611 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21acc972-2787-4279-84ff-bd5bfaf1f5ea None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:14:47.571522 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21acc972-2787-4279-84ff-bd5bfaf1f5ea None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:14:47.575048 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7243d2-f19f-4768-a9b8-756f961e1941 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:14:47.575640 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7243d2-f19f-4768-a9b8-756f961e1941 service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:14:47.575919 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7243d2-f19f-4768-a9b8-756f961e1941 service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:14:47.576188 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7243d2-f19f-4768-a9b8-756f961e1941 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:14:47.577939 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-cc7243d2-f19f-4768-a9b8-756f961e1941 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:14:47.639254 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7243d2-f19f-4768-a9b8-756f961e1941 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:14:47.640941 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7243d2-f19f-4768-a9b8-756f961e1941 service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:14:47.650479 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1314/2638] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:14:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:14:51.130628 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b15aa96-ae4d-4174-bfbc-8bfbfdedbe74 service cinder] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:14:51.132067 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-0b15aa96-ae4d-4174-bfbc-8bfbfdedbe74 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:14:51.210405 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b15aa96-ae4d-4174-bfbc-8bfbfdedbe74 service cinder] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:14:51.215503 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba684371-0adc-4348-a59e-12d2901bb0eb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2912307c56594125b157f1cfdf7c96a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:14:51.216110 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba684371-0adc-4348-a59e-12d2901bb0eb service placement] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:14:51.216392 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba684371-0adc-4348-a59e-12d2901bb0eb service placement] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:14:51.216658 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba684371-0adc-4348-a59e-12d2901bb0eb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:14:51.218497 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-ba684371-0adc-4348-a59e-12d2901bb0eb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:14:51.235933 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba684371-0adc-4348-a59e-12d2901bb0eb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:14:51.235933 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba684371-0adc-4348-a59e-12d2901bb0eb service placement] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:14:51.242076 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1325/2639] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:14:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:14:54.881179 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7243d2-f19f-4768-a9b8-756f961e1941 service neutron] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:14:54.893996 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7243d2-f19f-4768-a9b8-756f961e1941 service neutron] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:14:54.899373 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcbada7-32ab-4887-863b-d79182e15b4a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2912307c56594125b157f1cfdf7c96a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:14:54.899981 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcbada7-32ab-4887-863b-d79182e15b4a service placement] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:14:54.900259 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcbada7-32ab-4887-863b-d79182e15b4a service placement] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:14:54.900528 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcbada7-32ab-4887-863b-d79182e15b4a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:14:54.902425 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-cfcbada7-32ab-4887-863b-d79182e15b4a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:14:54.915172 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcbada7-32ab-4887-863b-d79182e15b4a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:14:54.916866 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcbada7-32ab-4887-863b-d79182e15b4a service placement] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:14:54.927314 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1315/2640] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:14:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:14:54.994281 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba684371-0adc-4348-a59e-12d2901bb0eb service placement] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:14:54.998344 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-ba684371-0adc-4348-a59e-12d2901bb0eb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:14:55.009492 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba684371-0adc-4348-a59e-12d2901bb0eb service placement] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:14:55.013932 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202fa2f7-c9fb-4f7e-a8e6-d04c2ef67bcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:14:55.014596 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202fa2f7-c9fb-4f7e-a8e6-d04c2ef67bcf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:14:55.014875 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202fa2f7-c9fb-4f7e-a8e6-d04c2ef67bcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:14:55.015149 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202fa2f7-c9fb-4f7e-a8e6-d04c2ef67bcf admin admin] PATH_INFO: `/v3/users/956ccf8932b24a7f8d74f4cbc3ead23a` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:14:55.016514 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202fa2f7-c9fb-4f7e-a8e6-d04c2ef67bcf admin admin] RBAC: Authorizing `identity:delete_user(user_id=956ccf8932b24a7f8d74f4cbc3ead23a)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:14:55.018467 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202fa2f7-c9fb-4f7e-a8e6-d04c2ef67bcf admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:14:55.297985 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-202fa2f7-c9fb-4f7e-a8e6-d04c2ef67bcf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '956ccf8932b24a7f8d74f4cbc3ead23a'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:14:55.309647 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-202fa2f7-c9fb-4f7e-a8e6-d04c2ef67bcf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '956ccf8932b24a7f8d74f4cbc3ead23a'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:14:55.316456 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-202fa2f7-c9fb-4f7e-a8e6-d04c2ef67bcf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '956ccf8932b24a7f8d74f4cbc3ead23a'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:14:55.369699 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-202fa2f7-c9fb-4f7e-a8e6-d04c2ef67bcf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '956ccf8932b24a7f8d74f4cbc3ead23a'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:14:55.370314 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-202fa2f7-c9fb-4f7e-a8e6-d04c2ef67bcf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '956ccf8932b24a7f8d74f4cbc3ead23a'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:14:55.376398 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1326/2641] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 09:14:54 2022] DELETE /identity/v3/users/956ccf8932b24a7f8d74f4cbc3ead23a => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:14:55.640652 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcbada7-32ab-4887-863b-d79182e15b4a service placement] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:14:55.651981 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcbada7-32ab-4887-863b-d79182e15b4a service placement] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:14:55.708246 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f2861b-dc41-4e1d-81d5-251bb707383a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:14:55.708865 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f2861b-dc41-4e1d-81d5-251bb707383a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:14:55.709172 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f2861b-dc41-4e1d-81d5-251bb707383a admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:14:55.709440 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f2861b-dc41-4e1d-81d5-251bb707383a admin admin] PATH_INFO: `/v3/projects/d1f83b4ed9ee42df9f946c9b4e13708d` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:14:55.710541 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f2861b-dc41-4e1d-81d5-251bb707383a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1f83b4ed9ee42df9f946c9b4e13708d)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:14:55.713600 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f2861b-dc41-4e1d-81d5-251bb707383a admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:14:55.797340 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:14:55.804114 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-d7f2861b-dc41-4e1d-81d5-251bb707383a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1f83b4ed9ee42df9f946c9b4e13708d', 'request_id': 'req-d7f2861b-dc41-4e1d-81d5-251bb707383a'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:14:55.844171 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-d7f2861b-dc41-4e1d-81d5-251bb707383a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1f83b4ed9ee42df9f946c9b4e13708d', 'request_id': 'req-d7f2861b-dc41-4e1d-81d5-251bb707383a'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:14:55.850005 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-d7f2861b-dc41-4e1d-81d5-251bb707383a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1f83b4ed9ee42df9f946c9b4e13708d', 'request_id': 'req-d7f2861b-dc41-4e1d-81d5-251bb707383a'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:14:55.862123 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-d7f2861b-dc41-4e1d-81d5-251bb707383a admin admin] The token cache is being invalidate because project d1f83b4ed9ee42df9f946c9b4e13708d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 09:14:55.862407 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-d7f2861b-dc41-4e1d-81d5-251bb707383a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:14:55.863045 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-d7f2861b-dc41-4e1d-81d5-251bb707383a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:14:55.864576 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1316/2642] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 09:14:55 2022] DELETE /identity/v3/projects/d1f83b4ed9ee42df9f946c9b4e13708d => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:14:55.868815 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202fa2f7-c9fb-4f7e-a8e6-d04c2ef67bcf admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:14:55.870251 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-202fa2f7-c9fb-4f7e-a8e6-d04c2ef67bcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:14:55.884426 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202fa2f7-c9fb-4f7e-a8e6-d04c2ef67bcf admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:14:55.944309 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c528770-bd32-42ab-a44e-b173dadab720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:14:55.944959 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c528770-bd32-42ab-a44e-b173dadab720 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:14:55.945244 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c528770-bd32-42ab-a44e-b173dadab720 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:14:55.946026 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c528770-bd32-42ab-a44e-b173dadab720 admin admin] PATH_INFO: `/v3/users/144656bc6b3b43d783632e10927dd5c4` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:14:55.948769 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:14:55.960085 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c528770-bd32-42ab-a44e-b173dadab720 admin admin] RBAC: Authorizing `identity:delete_user(user_id=144656bc6b3b43d783632e10927dd5c4)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:14:55.962788 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c528770-bd32-42ab-a44e-b173dadab720 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:14:56.119822 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-6c528770-bd32-42ab-a44e-b173dadab720 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '144656bc6b3b43d783632e10927dd5c4'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:14:56.131405 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-6c528770-bd32-42ab-a44e-b173dadab720 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '144656bc6b3b43d783632e10927dd5c4'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:14:56.138828 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-6c528770-bd32-42ab-a44e-b173dadab720 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '144656bc6b3b43d783632e10927dd5c4'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:14:56.203688 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-6c528770-bd32-42ab-a44e-b173dadab720 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '144656bc6b3b43d783632e10927dd5c4'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:14:56.204303 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-6c528770-bd32-42ab-a44e-b173dadab720 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '144656bc6b3b43d783632e10927dd5c4'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:14:56.210482 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1327/2643] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 09:14:55 2022] DELETE /identity/v3/users/144656bc6b3b43d783632e10927dd5c4 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:14:56.457870 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f2861b-dc41-4e1d-81d5-251bb707383a admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:14:56.459358 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-d7f2861b-dc41-4e1d-81d5-251bb707383a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:14:56.473419 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f2861b-dc41-4e1d-81d5-251bb707383a admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:14:56.530931 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387c461e-2d14-48b0-945b-6895dee37117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:14:56.531564 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387c461e-2d14-48b0-945b-6895dee37117 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:14:56.531839 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387c461e-2d14-48b0-945b-6895dee37117 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:14:56.532107 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387c461e-2d14-48b0-945b-6895dee37117 admin admin] PATH_INFO: `/v3/projects/80a657459f2b4b5db9f747c1e055ea77` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:14:56.542697 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387c461e-2d14-48b0-945b-6895dee37117 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80a657459f2b4b5db9f747c1e055ea77)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:14:56.544592 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387c461e-2d14-48b0-945b-6895dee37117 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:14:56.633932 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-387c461e-2d14-48b0-945b-6895dee37117 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80a657459f2b4b5db9f747c1e055ea77', 'request_id': 'req-387c461e-2d14-48b0-945b-6895dee37117'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:14:56.678197 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-387c461e-2d14-48b0-945b-6895dee37117 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80a657459f2b4b5db9f747c1e055ea77', 'request_id': 'req-387c461e-2d14-48b0-945b-6895dee37117'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:14:56.684308 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-387c461e-2d14-48b0-945b-6895dee37117 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80a657459f2b4b5db9f747c1e055ea77', 'request_id': 'req-387c461e-2d14-48b0-945b-6895dee37117'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:14:56.697086 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-387c461e-2d14-48b0-945b-6895dee37117 admin admin] The token cache is being invalidate because project 80a657459f2b4b5db9f747c1e055ea77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 09:14:56.697374 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-387c461e-2d14-48b0-945b-6895dee37117 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:14:56.698027 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-387c461e-2d14-48b0-945b-6895dee37117 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:14:56.699462 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1317/2644] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 09:14:56 2022] DELETE /identity/v3/projects/80a657459f2b4b5db9f747c1e055ea77 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:15:06.206821 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=30,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:15:06.693562 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=65,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:16:01.354285 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c528770-bd32-42ab-a44e-b173dadab720 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:16:01.355781 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-6c528770-bd32-42ab-a44e-b173dadab720 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:16:01.437540 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c528770-bd32-42ab-a44e-b173dadab720 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:16:01.497578 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7c7d3c-7c43-4f98-9272-7cb6b304955c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2912307c56594125b157f1cfdf7c96a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:16:01.498185 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7c7d3c-7c43-4f98-9272-7cb6b304955c service placement] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:16:01.498465 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7c7d3c-7c43-4f98-9272-7cb6b304955c service placement] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:16:01.498736 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7c7d3c-7c43-4f98-9272-7cb6b304955c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:16:01.500482 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-4a7c7d3c-7c43-4f98-9272-7cb6b304955c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:16:01.512257 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7c7d3c-7c43-4f98-9272-7cb6b304955c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:16:01.513834 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7c7d3c-7c43-4f98-9272-7cb6b304955c service placement] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:16:01.523962 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1328/2645] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:16:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:16:11.519755 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=63 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:17:38.017985 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387c461e-2d14-48b0-945b-6895dee37117 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:17:38.019407 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-387c461e-2d14-48b0-945b-6895dee37117 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:17:38.075239 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387c461e-2d14-48b0-945b-6895dee37117 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:17:38.078826 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bb587f-44cf-489c-89e4-19135afe9941 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ba74e28ced4cbba857f54222dfbe41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:17:38.079418 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bb587f-44cf-489c-89e4-19135afe9941 service cinder] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:17:38.079696 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bb587f-44cf-489c-89e4-19135afe9941 service cinder] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:17:38.079963 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bb587f-44cf-489c-89e4-19135afe9941 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:17:38.081704 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-47bb587f-44cf-489c-89e4-19135afe9941 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:17:38.159358 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bb587f-44cf-489c-89e4-19135afe9941 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:17:38.160778 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bb587f-44cf-489c-89e4-19135afe9941 service cinder] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:17:38.180022 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1318/2646] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:17:38 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:17:39.762903 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7c7d3c-7c43-4f98-9272-7cb6b304955c service placement] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:17:39.764554 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-4a7c7d3c-7c43-4f98-9272-7cb6b304955c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:17:39.821015 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7c7d3c-7c43-4f98-9272-7cb6b304955c service placement] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:17:39.824527 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525936a2-8647-4037-a42e-9d3e1074bc63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:17:39.825142 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525936a2-8647-4037-a42e-9d3e1074bc63 service neutron] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:17:39.825417 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525936a2-8647-4037-a42e-9d3e1074bc63 service neutron] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:17:39.825681 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525936a2-8647-4037-a42e-9d3e1074bc63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:17:39.827454 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-525936a2-8647-4037-a42e-9d3e1074bc63 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:17:39.909263 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525936a2-8647-4037-a42e-9d3e1074bc63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:17:39.910734 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525936a2-8647-4037-a42e-9d3e1074bc63 service neutron] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:17:39.929670 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1329/2647] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:17:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:17:43.352414 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bb587f-44cf-489c-89e4-19135afe9941 service cinder] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:17:43.353969 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-47bb587f-44cf-489c-89e4-19135afe9941 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:17:43.370685 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bb587f-44cf-489c-89e4-19135afe9941 service cinder] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:17:43.375418 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b36df63-57d2-4cea-90b9-1b5ab8087e06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:17:43.376063 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b36df63-57d2-4cea-90b9-1b5ab8087e06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:17:43.376342 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b36df63-57d2-4cea-90b9-1b5ab8087e06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:17:43.376613 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b36df63-57d2-4cea-90b9-1b5ab8087e06 admin admin] PATH_INFO: `/v3/users/258ec0f975eb47738f4dac356c0578e6` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:17:43.391293 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b36df63-57d2-4cea-90b9-1b5ab8087e06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=258ec0f975eb47738f4dac356c0578e6)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:17:43.393178 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b36df63-57d2-4cea-90b9-1b5ab8087e06 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:17:43.596165 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-1b36df63-57d2-4cea-90b9-1b5ab8087e06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '258ec0f975eb47738f4dac356c0578e6'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:17:43.608561 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-1b36df63-57d2-4cea-90b9-1b5ab8087e06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '258ec0f975eb47738f4dac356c0578e6'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:17:43.613014 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:17:43.615819 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-1b36df63-57d2-4cea-90b9-1b5ab8087e06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '258ec0f975eb47738f4dac356c0578e6'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:17:43.616499 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-1b36df63-57d2-4cea-90b9-1b5ab8087e06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '258ec0f975eb47738f4dac356c0578e6'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:17:43.617137 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-1b36df63-57d2-4cea-90b9-1b5ab8087e06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '258ec0f975eb47738f4dac356c0578e6'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:17:43.701704 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1319/2648] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 09:17:43 2022] DELETE /identity/v3/users/258ec0f975eb47738f4dac356c0578e6 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:17:43.943508 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525936a2-8647-4037-a42e-9d3e1074bc63 service neutron] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:17:43.955064 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525936a2-8647-4037-a42e-9d3e1074bc63 service neutron] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:17:44.021339 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c940c9d0-dc6b-4c39-ad3a-481ba6deefcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:17:44.021958 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c940c9d0-dc6b-4c39-ad3a-481ba6deefcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:17:44.022234 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c940c9d0-dc6b-4c39-ad3a-481ba6deefcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:17:44.022501 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c940c9d0-dc6b-4c39-ad3a-481ba6deefcd admin admin] PATH_INFO: `/v3/projects/e7ab7cc7a6364f5bad3ca36b070968f8` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:17:44.033738 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c940c9d0-dc6b-4c39-ad3a-481ba6deefcd admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7ab7cc7a6364f5bad3ca36b070968f8)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:17:44.035631 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c940c9d0-dc6b-4c39-ad3a-481ba6deefcd admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:17:44.050939 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:17:44.165974 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-c940c9d0-dc6b-4c39-ad3a-481ba6deefcd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7ab7cc7a6364f5bad3ca36b070968f8', 'request_id': 'req-c940c9d0-dc6b-4c39-ad3a-481ba6deefcd'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:17:44.171607 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-c940c9d0-dc6b-4c39-ad3a-481ba6deefcd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7ab7cc7a6364f5bad3ca36b070968f8', 'request_id': 'req-c940c9d0-dc6b-4c39-ad3a-481ba6deefcd'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:17:44.244547 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-c940c9d0-dc6b-4c39-ad3a-481ba6deefcd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7ab7cc7a6364f5bad3ca36b070968f8', 'request_id': 'req-c940c9d0-dc6b-4c39-ad3a-481ba6deefcd'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:17:44.257663 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-c940c9d0-dc6b-4c39-ad3a-481ba6deefcd admin admin] The token cache is being invalidate because project e7ab7cc7a6364f5bad3ca36b070968f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 09:17:44.260035 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-c940c9d0-dc6b-4c39-ad3a-481ba6deefcd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:17:44.260035 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-c940c9d0-dc6b-4c39-ad3a-481ba6deefcd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:17:44.260296 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1330/2649] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 09:17:43 2022] DELETE /identity/v3/projects/e7ab7cc7a6364f5bad3ca36b070968f8 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:17:44.266318 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e8009c-f05d-4b59-8426-a253c3937c3f None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:17:44.266649 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e8009c-f05d-4b59-8426-a253c3937c3f None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:17:44.266917 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e8009c-f05d-4b59-8426-a253c3937c3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:17:44.324253 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-11e8009c-f05d-4b59-8426-a253c3937c3f None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:17:44.325486 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-11e8009c-f05d-4b59-8426-a253c3937c3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:17:44.387681 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1320/2650] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 09:17:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:17:44.392224 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb6ca1b-26e8-4b0b-8292-6934e0941765 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:17:44.392512 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb6ca1b-26e8-4b0b-8292-6934e0941765 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:17:44.392783 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb6ca1b-26e8-4b0b-8292-6934e0941765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:17:44.412778 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-5bb6ca1b-26e8-4b0b-8292-6934e0941765 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:17:44.414323 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-5bb6ca1b-26e8-4b0b-8292-6934e0941765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:17:44.422420 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1331/2651] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 09:17:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:17:44.426798 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e8009c-f05d-4b59-8426-a253c3937c3f None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:17:44.437813 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e8009c-f05d-4b59-8426-a253c3937c3f None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:17:44.442740 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a4d11e-5ae9-47b0-b892-ce3414e7601b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:17:44.443378 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a4d11e-5ae9-47b0-b892-ce3414e7601b admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:17:44.443658 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a4d11e-5ae9-47b0-b892-ce3414e7601b admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:17:44.443921 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a4d11e-5ae9-47b0-b892-ce3414e7601b admin admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:17:44.445477 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a4d11e-5ae9-47b0-b892-ce3414e7601b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:17:44.445792 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a4d11e-5ae9-47b0-b892-ce3414e7601b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:17:44.447389 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a4d11e-5ae9-47b0-b892-ce3414e7601b admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:17:44.459393 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1321/2652] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Aug 10 09:17:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:17:44.463860 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb6ca1b-26e8-4b0b-8292-6934e0941765 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:17:44.473265 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb6ca1b-26e8-4b0b-8292-6934e0941765 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:17:44.478039 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b63137-b33f-44b4-b2f9-4d3338764b84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:17:44.478642 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b63137-b33f-44b4-b2f9-4d3338764b84 admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:17:44.478918 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b63137-b33f-44b4-b2f9-4d3338764b84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:17:44.479185 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b63137-b33f-44b4-b2f9-4d3338764b84 admin admin] PATH_INFO: `/v3/projects` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:17:44.479802 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b63137-b33f-44b4-b2f9-4d3338764b84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:17:44.481751 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b63137-b33f-44b4-b2f9-4d3338764b84 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:17:44.578553 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1332/2653] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 09:17:44 2022] POST /identity/v3/projects => generated 369 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 09:17:44.582663 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a4d11e-5ae9-47b0-b892-ce3414e7601b admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:17:44.592860 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a4d11e-5ae9-47b0-b892-ce3414e7601b admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:17:44.656046 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc274748-2bfc-499d-8f40-fe852ff7b330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:17:44.656872 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc274748-2bfc-499d-8f40-fe852ff7b330 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:17:44.657279 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc274748-2bfc-499d-8f40-fe852ff7b330 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:17:44.657759 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc274748-2bfc-499d-8f40-fe852ff7b330 admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:17:44.658920 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc274748-2bfc-499d-8f40-fe852ff7b330 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:17:44.662590 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc274748-2bfc-499d-8f40-fe852ff7b330 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:17:44.740051 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1322/2654] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 09:17:44 2022] POST /identity/v3/users => generated 285 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 09:17:44.744865 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b63137-b33f-44b4-b2f9-4d3338764b84 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:17:44.755611 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b63137-b33f-44b4-b2f9-4d3338764b84 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:17:44.760582 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23843323-b503-4fd0-8c09-fd522c2ee681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:17:44.761217 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23843323-b503-4fd0-8c09-fd522c2ee681 admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:17:44.761524 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23843323-b503-4fd0-8c09-fd522c2ee681 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:17:44.761813 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23843323-b503-4fd0-8c09-fd522c2ee681 admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:17:44.762446 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23843323-b503-4fd0-8c09-fd522c2ee681 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:17:44.764025 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23843323-b503-4fd0-8c09-fd522c2ee681 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:17:44.774171 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1333/2655] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 09:17:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 09:17:44.778155 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc274748-2bfc-499d-8f40-fe852ff7b330 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:17:44.787953 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc274748-2bfc-499d-8f40-fe852ff7b330 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:17:44.793977 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0250d309-7023-4bf7-b462-b1ceb2d5adb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:17:44.794590 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0250d309-7023-4bf7-b462-b1ceb2d5adb9 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:17:44.794870 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0250d309-7023-4bf7-b462-b1ceb2d5adb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:17:44.795143 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0250d309-7023-4bf7-b462-b1ceb2d5adb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:17:44.795787 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0250d309-7023-4bf7-b462-b1ceb2d5adb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:17:44.797451 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0250d309-7023-4bf7-b462-b1ceb2d5adb9 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:17:44.808761 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1323/2656] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 09:17:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 09:17:44.812716 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23843323-b503-4fd0-8c09-fd522c2ee681 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:17:44.822688 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23843323-b503-4fd0-8c09-fd522c2ee681 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:17:44.828760 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fe28e2-c3d3-42e0-8663-7524277407c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:17:44.828760 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fe28e2-c3d3-42e0-8663-7524277407c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:17:44.828760 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fe28e2-c3d3-42e0-8663-7524277407c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:17:44.829161 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fe28e2-c3d3-42e0-8663-7524277407c2 admin admin] PATH_INFO: `/v3/projects/11ac43eb3ba24931abfac99a1f1eb67c/users/2a6365dc52f0402db08b151014dbb22e/roles/44b64db484f24a46bf0b7ced46b519bf` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:17:44.844408 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fe28e2-c3d3-42e0-8663-7524277407c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11ac43eb3ba24931abfac99a1f1eb67c, user_id=2a6365dc52f0402db08b151014dbb22e, role_id=44b64db484f24a46bf0b7ced46b519bf)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:17:44.847410 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fe28e2-c3d3-42e0-8663-7524277407c2 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:17:44.923015 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1334/2657] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 09:17:44 2022] PUT /identity/v3/projects/11ac43eb3ba24931abfac99a1f1eb67c/users/2a6365dc52f0402db08b151014dbb22e/roles/44b64db484f24a46bf0b7ced46b519bf => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:17:44.988431 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95c8be1-79b8-4f28-901e-a59ee292a62a None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:17:44.988643 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95c8be1-79b8-4f28-901e-a59ee292a62a None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:17:44.988890 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95c8be1-79b8-4f28-901e-a59ee292a62a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:17:45.016850 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-d95c8be1-79b8-4f28-901e-a59ee292a62a None None] MFA Rules not processed for user `2a6365dc52f0402db08b151014dbb22e`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:17:45.018092 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-d95c8be1-79b8-4f28-901e-a59ee292a62a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:17:45.027723 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:17:45.082865 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1324/2658] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 09:17:44 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:17:45.087320 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fe28e2-c3d3-42e0-8663-7524277407c2 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:17:45.147627 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fe28e2-c3d3-42e0-8663-7524277407c2 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:17:45.152556 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4acdbb-13a0-423f-92a8-45f86deae1a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:17:45.153186 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4acdbb-13a0-423f-92a8-45f86deae1a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:17:45.153462 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4acdbb-13a0-423f-92a8-45f86deae1a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:17:45.153730 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4acdbb-13a0-423f-92a8-45f86deae1a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:17:45.154357 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4acdbb-13a0-423f-92a8-45f86deae1a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:17:45.156240 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4acdbb-13a0-423f-92a8-45f86deae1a2 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:17:45.234626 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1335/2659] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 09:17:45 2022] POST /identity/v3/projects => generated 373 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 09:17:45.238782 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95c8be1-79b8-4f28-901e-a59ee292a62a None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:17:45.249969 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95c8be1-79b8-4f28-901e-a59ee292a62a None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:17:45.307063 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76b61bb-e6df-48e6-8116-80194d600f55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:17:45.307654 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76b61bb-e6df-48e6-8116-80194d600f55 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:17:45.307929 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76b61bb-e6df-48e6-8116-80194d600f55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:17:45.308194 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76b61bb-e6df-48e6-8116-80194d600f55 admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:17:45.308796 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76b61bb-e6df-48e6-8116-80194d600f55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:17:45.310710 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76b61bb-e6df-48e6-8116-80194d600f55 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:17:45.439813 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1325/2660] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 09:17:45 2022] POST /identity/v3/users => generated 293 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 09:17:45.443771 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4acdbb-13a0-423f-92a8-45f86deae1a2 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:17:45.454798 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4acdbb-13a0-423f-92a8-45f86deae1a2 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:17:45.459198 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696987ea-7cf2-4618-9369-a6eec70747b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:17:45.459787 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696987ea-7cf2-4618-9369-a6eec70747b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:17:45.460061 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696987ea-7cf2-4618-9369-a6eec70747b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:17:45.460845 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696987ea-7cf2-4618-9369-a6eec70747b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:17:45.461512 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696987ea-7cf2-4618-9369-a6eec70747b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:17:45.463076 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696987ea-7cf2-4618-9369-a6eec70747b1 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:17:45.473110 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1336/2661] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 09:17:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 09:17:45.477652 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76b61bb-e6df-48e6-8116-80194d600f55 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:17:45.487403 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76b61bb-e6df-48e6-8116-80194d600f55 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:17:45.492572 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9e02b2-082b-4805-9470-53962cdacf1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:17:45.493201 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9e02b2-082b-4805-9470-53962cdacf1b admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:17:45.493477 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9e02b2-082b-4805-9470-53962cdacf1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:17:45.493743 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9e02b2-082b-4805-9470-53962cdacf1b admin admin] PATH_INFO: `/v3/projects/14a7579e39034285b1837874e4df7123/users/a01c5fd94e474195862db8e73d88deea/roles/ec3751692d9c4ecf81b4a71c936c46be` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:17:45.508558 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9e02b2-082b-4805-9470-53962cdacf1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=14a7579e39034285b1837874e4df7123, user_id=a01c5fd94e474195862db8e73d88deea, role_id=ec3751692d9c4ecf81b4a71c936c46be)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:17:45.511205 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9e02b2-082b-4805-9470-53962cdacf1b admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:17:45.582586 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1326/2662] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 09:17:45 2022] PUT /identity/v3/projects/14a7579e39034285b1837874e4df7123/users/a01c5fd94e474195862db8e73d88deea/roles/ec3751692d9c4ecf81b4a71c936c46be => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:17:45.590027 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696987ea-7cf2-4618-9369-a6eec70747b1 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:17:45.596264 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:17:45.599963 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696987ea-7cf2-4618-9369-a6eec70747b1 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:17:45.654624 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05d322e-d3c8-45ea-998d-4b25b82017c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:17:45.655225 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05d322e-d3c8-45ea-998d-4b25b82017c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:17:45.655507 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05d322e-d3c8-45ea-998d-4b25b82017c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:17:45.655805 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05d322e-d3c8-45ea-998d-4b25b82017c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:17:45.656437 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05d322e-d3c8-45ea-998d-4b25b82017c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:17:45.658064 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05d322e-d3c8-45ea-998d-4b25b82017c1 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:17:45.668387 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1337/2663] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 09:17:45 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 09:17:45.672594 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9e02b2-082b-4805-9470-53962cdacf1b admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:17:45.683528 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9e02b2-082b-4805-9470-53962cdacf1b admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:17:45.688131 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29c4f2d-cdb7-4320-b9a2-2a0fa36a3adc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:17:45.688776 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29c4f2d-cdb7-4320-b9a2-2a0fa36a3adc admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:17:45.689158 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29c4f2d-cdb7-4320-b9a2-2a0fa36a3adc admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:17:45.689435 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29c4f2d-cdb7-4320-b9a2-2a0fa36a3adc admin admin] PATH_INFO: `/v3/projects/14a7579e39034285b1837874e4df7123/users/a01c5fd94e474195862db8e73d88deea/roles/ec3751692d9c4ecf81b4a71c936c46be` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:17:45.691481 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29c4f2d-cdb7-4320-b9a2-2a0fa36a3adc admin admin] RBAC: Authorizing `identity:create_grant(project_id=14a7579e39034285b1837874e4df7123, user_id=a01c5fd94e474195862db8e73d88deea, role_id=ec3751692d9c4ecf81b4a71c936c46be)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:17:45.694202 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29c4f2d-cdb7-4320-b9a2-2a0fa36a3adc admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:17:45.704879 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1327/2664] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 09:17:45 2022] PUT /identity/v3/projects/14a7579e39034285b1837874e4df7123/users/a01c5fd94e474195862db8e73d88deea/roles/ec3751692d9c4ecf81b4a71c936c46be => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:17:45.714727 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3047daba-0d09-4a3c-8b7f-869f9d3a6879 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:17:45.715008 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3047daba-0d09-4a3c-8b7f-869f9d3a6879 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:17:45.715275 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3047daba-0d09-4a3c-8b7f-869f9d3a6879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:17:45.743584 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-3047daba-0d09-4a3c-8b7f-869f9d3a6879 None None] MFA Rules not processed for user `a01c5fd94e474195862db8e73d88deea`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:17:45.744753 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-3047daba-0d09-4a3c-8b7f-869f9d3a6879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:17:45.813037 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1338/2665] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 09:17:45 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:17:45.824058 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29c4f2d-cdb7-4320-b9a2-2a0fa36a3adc admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:17:45.825756 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-c29c4f2d-cdb7-4320-b9a2-2a0fa36a3adc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:17:45.882730 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29c4f2d-cdb7-4320-b9a2-2a0fa36a3adc admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:17:45.886370 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212844b6-5088-48f8-b763-a4471eeacfc9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ba74e28ced4cbba857f54222dfbe41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:17:45.886961 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212844b6-5088-48f8-b763-a4471eeacfc9 service cinder] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:17:45.887245 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212844b6-5088-48f8-b763-a4471eeacfc9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:17:45.887516 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212844b6-5088-48f8-b763-a4471eeacfc9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:17:45.896722 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212844b6-5088-48f8-b763-a4471eeacfc9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:17:45.898719 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212844b6-5088-48f8-b763-a4471eeacfc9 service cinder] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:17:45.908094 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1328/2666] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:17:45 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:17:46.205292 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3047daba-0d09-4a3c-8b7f-869f9d3a6879 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:17:46.215516 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3047daba-0d09-4a3c-8b7f-869f9d3a6879 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:17:46.219048 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705454df-cbd2-41bc-a293-ebe94cbcf34c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ba74e28ced4cbba857f54222dfbe41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:17:46.219639 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705454df-cbd2-41bc-a293-ebe94cbcf34c service cinder] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:17:46.219932 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705454df-cbd2-41bc-a293-ebe94cbcf34c service cinder] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:17:46.220186 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705454df-cbd2-41bc-a293-ebe94cbcf34c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:17:46.276562 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705454df-cbd2-41bc-a293-ebe94cbcf34c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:17:46.278016 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705454df-cbd2-41bc-a293-ebe94cbcf34c service cinder] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:17:46.286574 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1339/2667] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:17:46 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:17:55.903908 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=92,keystone:INSERT=5 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:17:56.282656 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=89 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:18:58.260736 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212844b6-5088-48f8-b763-a4471eeacfc9 service cinder] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:18:58.269553 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212844b6-5088-48f8-b763-a4471eeacfc9 service cinder] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:18:58.273415 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c98243d-7fe4-46ae-b31e-ac05e467375c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ba74e28ced4cbba857f54222dfbe41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:18:58.274002 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c98243d-7fe4-46ae-b31e-ac05e467375c service cinder] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:18:58.274280 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c98243d-7fe4-46ae-b31e-ac05e467375c service cinder] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:18:58.274547 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c98243d-7fe4-46ae-b31e-ac05e467375c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:18:58.276232 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-8c98243d-7fe4-46ae-b31e-ac05e467375c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:18:58.333254 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c98243d-7fe4-46ae-b31e-ac05e467375c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:18:58.334602 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c98243d-7fe4-46ae-b31e-ac05e467375c service cinder] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:18:58.343907 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1329/2668] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:18:58 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:19:08.339348 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=28 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:21:07.368588 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705454df-cbd2-41bc-a293-ebe94cbcf34c service cinder] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:21:07.370544 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-705454df-cbd2-41bc-a293-ebe94cbcf34c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:21:07.480578 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705454df-cbd2-41bc-a293-ebe94cbcf34c service cinder] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:21:07.485647 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53237ad-b208-4088-9634-2e2449b03443 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2912307c56594125b157f1cfdf7c96a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:21:07.486248 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53237ad-b208-4088-9634-2e2449b03443 service placement] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:21:07.486526 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53237ad-b208-4088-9634-2e2449b03443 service placement] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:21:07.486793 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53237ad-b208-4088-9634-2e2449b03443 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:21:07.488507 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-e53237ad-b208-4088-9634-2e2449b03443 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:21:07.500328 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53237ad-b208-4088-9634-2e2449b03443 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:21:07.501919 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53237ad-b208-4088-9634-2e2449b03443 service placement] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:21:07.512069 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1340/2669] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:21:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:21:17.507595 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=49 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:21:37.523676 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c98243d-7fe4-46ae-b31e-ac05e467375c service cinder] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:21:37.525136 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-8c98243d-7fe4-46ae-b31e-ac05e467375c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:21:37.546799 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c98243d-7fe4-46ae-b31e-ac05e467375c service cinder] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:21:37.550399 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8b37c7-f291-461b-812a-2481ab2ebc25 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ba74e28ced4cbba857f54222dfbe41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:21:37.550985 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8b37c7-f291-461b-812a-2481ab2ebc25 service cinder] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:21:37.551257 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8b37c7-f291-461b-812a-2481ab2ebc25 service cinder] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:21:37.551575 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8b37c7-f291-461b-812a-2481ab2ebc25 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:21:37.553220 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-cf8b37c7-f291-461b-812a-2481ab2ebc25 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:21:37.617863 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8b37c7-f291-461b-812a-2481ab2ebc25 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:21:37.619344 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8b37c7-f291-461b-812a-2481ab2ebc25 service cinder] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:21:37.630047 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1330/2670] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:21:37 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:21:47.625814 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=34 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:21:54.253826 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53237ad-b208-4088-9634-2e2449b03443 service placement] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:21:54.258241 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-e53237ad-b208-4088-9634-2e2449b03443 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:21:54.313171 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53237ad-b208-4088-9634-2e2449b03443 service placement] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:21:54.316728 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e34616-01d6-4e1b-b5ef-8a3ab13b1040 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:21:54.317420 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e34616-01d6-4e1b-b5ef-8a3ab13b1040 service neutron] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:21:54.317701 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e34616-01d6-4e1b-b5ef-8a3ab13b1040 service neutron] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:21:54.317973 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e34616-01d6-4e1b-b5ef-8a3ab13b1040 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:21:54.319786 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-23e34616-01d6-4e1b-b5ef-8a3ab13b1040 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:21:54.384709 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e34616-01d6-4e1b-b5ef-8a3ab13b1040 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:21:54.386435 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e34616-01d6-4e1b-b5ef-8a3ab13b1040 service neutron] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:21:54.396390 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1341/2671] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:21:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:21:54.903965 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8b37c7-f291-461b-812a-2481ab2ebc25 service cinder] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:21:54.905465 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-cf8b37c7-f291-461b-812a-2481ab2ebc25 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:21:54.920397 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8b37c7-f291-461b-812a-2481ab2ebc25 service cinder] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:21:54.925432 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b772df5a-d2dd-4af4-a831-6eeb27cfd60d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2912307c56594125b157f1cfdf7c96a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:21:54.926030 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b772df5a-d2dd-4af4-a831-6eeb27cfd60d service placement] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:21:54.926307 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b772df5a-d2dd-4af4-a831-6eeb27cfd60d service placement] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:21:54.926572 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b772df5a-d2dd-4af4-a831-6eeb27cfd60d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:21:54.929013 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-b772df5a-d2dd-4af4-a831-6eeb27cfd60d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:21:54.940796 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b772df5a-d2dd-4af4-a831-6eeb27cfd60d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:21:54.942492 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b772df5a-d2dd-4af4-a831-6eeb27cfd60d service placement] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:21:54.952293 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1331/2672] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:21:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:21:55.528303 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e34616-01d6-4e1b-b5ef-8a3ab13b1040 service neutron] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:21:55.541272 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e34616-01d6-4e1b-b5ef-8a3ab13b1040 service neutron] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:21:55.546208 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785427e9-447a-4c0c-85b6-531e8763a8f2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2912307c56594125b157f1cfdf7c96a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:21:55.547325 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785427e9-447a-4c0c-85b6-531e8763a8f2 service placement] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:21:55.547325 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785427e9-447a-4c0c-85b6-531e8763a8f2 service placement] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:21:55.547615 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785427e9-447a-4c0c-85b6-531e8763a8f2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:21:55.549384 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-785427e9-447a-4c0c-85b6-531e8763a8f2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:21:55.561429 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785427e9-447a-4c0c-85b6-531e8763a8f2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:21:55.563194 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785427e9-447a-4c0c-85b6-531e8763a8f2 service placement] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:21:55.577808 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1342/2673] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:21:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:21:55.659418 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b772df5a-d2dd-4af4-a831-6eeb27cfd60d service placement] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:21:55.662452 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-b772df5a-d2dd-4af4-a831-6eeb27cfd60d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:21:55.678346 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b772df5a-d2dd-4af4-a831-6eeb27cfd60d service placement] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:21:55.682998 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff02d22-5554-4855-8da1-7914515f8b90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:21:55.683667 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff02d22-5554-4855-8da1-7914515f8b90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:21:55.683940 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff02d22-5554-4855-8da1-7914515f8b90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:21:55.684207 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff02d22-5554-4855-8da1-7914515f8b90 admin admin] PATH_INFO: `/v3/users/84219759abdb4509aef31cbc4dcbdb08` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:21:55.685333 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff02d22-5554-4855-8da1-7914515f8b90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84219759abdb4509aef31cbc4dcbdb08)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:21:55.687225 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff02d22-5554-4855-8da1-7914515f8b90 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:21:55.858111 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-7ff02d22-5554-4855-8da1-7914515f8b90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84219759abdb4509aef31cbc4dcbdb08'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:21:55.869730 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-7ff02d22-5554-4855-8da1-7914515f8b90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84219759abdb4509aef31cbc4dcbdb08'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:21:55.876415 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-7ff02d22-5554-4855-8da1-7914515f8b90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84219759abdb4509aef31cbc4dcbdb08'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:21:55.877124 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-7ff02d22-5554-4855-8da1-7914515f8b90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84219759abdb4509aef31cbc4dcbdb08'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:21:55.877750 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-7ff02d22-5554-4855-8da1-7914515f8b90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84219759abdb4509aef31cbc4dcbdb08'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:21:55.941622 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1332/2674] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 09:21:55 2022] DELETE /identity/v3/users/84219759abdb4509aef31cbc4dcbdb08 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:21:56.184601 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785427e9-447a-4c0c-85b6-531e8763a8f2 service placement] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:21:56.200182 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785427e9-447a-4c0c-85b6-531e8763a8f2 service placement] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:21:56.270460 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bc9ff2-5d92-4838-a833-9c3b599b7454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:21:56.271011 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bc9ff2-5d92-4838-a833-9c3b599b7454 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:21:56.271287 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bc9ff2-5d92-4838-a833-9c3b599b7454 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:21:56.271560 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bc9ff2-5d92-4838-a833-9c3b599b7454 admin admin] PATH_INFO: `/v3/projects/6b2c0dc9dfab4bad8e6dd81cebf6a16f` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:21:56.273305 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bc9ff2-5d92-4838-a833-9c3b599b7454 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b2c0dc9dfab4bad8e6dd81cebf6a16f)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:21:56.275190 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bc9ff2-5d92-4838-a833-9c3b599b7454 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:21:56.378152 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:21:56.385754 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-01bc9ff2-5d92-4838-a833-9c3b599b7454 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b2c0dc9dfab4bad8e6dd81cebf6a16f', 'request_id': 'req-01bc9ff2-5d92-4838-a833-9c3b599b7454'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:21:56.391437 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-01bc9ff2-5d92-4838-a833-9c3b599b7454 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b2c0dc9dfab4bad8e6dd81cebf6a16f', 'request_id': 'req-01bc9ff2-5d92-4838-a833-9c3b599b7454'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:21:56.444969 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-01bc9ff2-5d92-4838-a833-9c3b599b7454 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b2c0dc9dfab4bad8e6dd81cebf6a16f', 'request_id': 'req-01bc9ff2-5d92-4838-a833-9c3b599b7454'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:21:56.456341 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-01bc9ff2-5d92-4838-a833-9c3b599b7454 admin admin] The token cache is being invalidate because project 6b2c0dc9dfab4bad8e6dd81cebf6a16f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 09:21:56.456624 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-01bc9ff2-5d92-4838-a833-9c3b599b7454 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:21:56.457248 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-01bc9ff2-5d92-4838-a833-9c3b599b7454 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:21:56.458704 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1343/2675] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 09:21:56 2022] DELETE /identity/v3/projects/6b2c0dc9dfab4bad8e6dd81cebf6a16f => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 09:21:56.463387 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff02d22-5554-4855-8da1-7914515f8b90 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:21:56.464789 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-7ff02d22-5554-4855-8da1-7914515f8b90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:21:56.478203 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff02d22-5554-4855-8da1-7914515f8b90 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:21:56.534832 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f28c54b-7f05-45c6-afb7-2c48eff555b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:21:56.535454 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f28c54b-7f05-45c6-afb7-2c48eff555b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:21:56.535733 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f28c54b-7f05-45c6-afb7-2c48eff555b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:21:56.536006 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f28c54b-7f05-45c6-afb7-2c48eff555b4 admin admin] PATH_INFO: `/v3/users/1a655a3e5aef453a86cf8e8ff39f423b` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:21:56.537450 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f28c54b-7f05-45c6-afb7-2c48eff555b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a655a3e5aef453a86cf8e8ff39f423b)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:21:56.539329 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f28c54b-7f05-45c6-afb7-2c48eff555b4 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:21:56.680659 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-9f28c54b-7f05-45c6-afb7-2c48eff555b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a655a3e5aef453a86cf8e8ff39f423b'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:21:56.690690 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:21:56.692890 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-9f28c54b-7f05-45c6-afb7-2c48eff555b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a655a3e5aef453a86cf8e8ff39f423b'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:21:56.700351 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-9f28c54b-7f05-45c6-afb7-2c48eff555b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a655a3e5aef453a86cf8e8ff39f423b'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:21:56.701080 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-9f28c54b-7f05-45c6-afb7-2c48eff555b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a655a3e5aef453a86cf8e8ff39f423b'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:21:56.701691 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-9f28c54b-7f05-45c6-afb7-2c48eff555b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a655a3e5aef453a86cf8e8ff39f423b'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:21:56.770332 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1333/2676] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 09:21:56 2022] DELETE /identity/v3/users/1a655a3e5aef453a86cf8e8ff39f423b => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:21:57.017108 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bc9ff2-5d92-4838-a833-9c3b599b7454 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:21:57.018751 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-01bc9ff2-5d92-4838-a833-9c3b599b7454 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:21:57.031604 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bc9ff2-5d92-4838-a833-9c3b599b7454 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:21:57.091150 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aee105-c6d7-4d97-9fa1-1e0b5415e2d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:21:57.092367 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aee105-c6d7-4d97-9fa1-1e0b5415e2d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:21:57.092367 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aee105-c6d7-4d97-9fa1-1e0b5415e2d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:21:57.092367 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aee105-c6d7-4d97-9fa1-1e0b5415e2d6 admin admin] PATH_INFO: `/v3/projects/90ec6a9ff8374dc3913265622034ddf8` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:21:57.093460 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7aee105-c6d7-4d97-9fa1-1e0b5415e2d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90ec6a9ff8374dc3913265622034ddf8)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:21:57.095379 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7aee105-c6d7-4d97-9fa1-1e0b5415e2d6 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:21:57.230378 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-e7aee105-c6d7-4d97-9fa1-1e0b5415e2d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90ec6a9ff8374dc3913265622034ddf8', 'request_id': 'req-e7aee105-c6d7-4d97-9fa1-1e0b5415e2d6'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:21:57.236129 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-e7aee105-c6d7-4d97-9fa1-1e0b5415e2d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90ec6a9ff8374dc3913265622034ddf8', 'request_id': 'req-e7aee105-c6d7-4d97-9fa1-1e0b5415e2d6'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:21:57.313373 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-e7aee105-c6d7-4d97-9fa1-1e0b5415e2d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90ec6a9ff8374dc3913265622034ddf8', 'request_id': 'req-e7aee105-c6d7-4d97-9fa1-1e0b5415e2d6'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:21:57.325333 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-e7aee105-c6d7-4d97-9fa1-1e0b5415e2d6 admin admin] The token cache is being invalidate because project 90ec6a9ff8374dc3913265622034ddf8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 09:21:57.325624 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-e7aee105-c6d7-4d97-9fa1-1e0b5415e2d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:21:57.326275 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-e7aee105-c6d7-4d97-9fa1-1e0b5415e2d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:21:57.327757 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1344/2677] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 09:21:57 2022] DELETE /identity/v3/projects/90ec6a9ff8374dc3913265622034ddf8 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:21:57.332864 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78677aad-d555-4a04-98bd-d55b71be243d None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:21:57.333349 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78677aad-d555-4a04-98bd-d55b71be243d None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:21:57.333441 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78677aad-d555-4a04-98bd-d55b71be243d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:21:57.354208 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-78677aad-d555-4a04-98bd-d55b71be243d None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:21:57.355651 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-78677aad-d555-4a04-98bd-d55b71be243d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:21:57.419532 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1334/2678] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 09:21:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:21:57.432148 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f367309-89b7-4138-a937-5e482efc35c9 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:21:57.432427 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f367309-89b7-4138-a937-5e482efc35c9 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:21:57.432718 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f367309-89b7-4138-a937-5e482efc35c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:21:57.452810 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-8f367309-89b7-4138-a937-5e482efc35c9 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:21:57.454225 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-8f367309-89b7-4138-a937-5e482efc35c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:21:57.462469 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1345/2679] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 09:21:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:21:57.466683 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78677aad-d555-4a04-98bd-d55b71be243d None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:21:57.476716 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78677aad-d555-4a04-98bd-d55b71be243d None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:21:57.481217 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a103b9ae-d33a-40b7-97ce-e328c048f267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:21:57.481802 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a103b9ae-d33a-40b7-97ce-e328c048f267 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:21:57.482078 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a103b9ae-d33a-40b7-97ce-e328c048f267 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:21:57.482368 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a103b9ae-d33a-40b7-97ce-e328c048f267 admin admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:21:57.482980 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a103b9ae-d33a-40b7-97ce-e328c048f267 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:21:57.483279 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a103b9ae-d33a-40b7-97ce-e328c048f267 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:21:57.484842 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a103b9ae-d33a-40b7-97ce-e328c048f267 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:21:57.499531 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1335/2680] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Aug 10 09:21:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:21:57.503917 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f367309-89b7-4138-a937-5e482efc35c9 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:21:57.514224 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f367309-89b7-4138-a937-5e482efc35c9 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:21:57.519056 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c515efb-366d-4759-8d75-222d65b68bab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:21:57.519600 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c515efb-366d-4759-8d75-222d65b68bab admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:21:57.519877 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c515efb-366d-4759-8d75-222d65b68bab admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:21:57.520144 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c515efb-366d-4759-8d75-222d65b68bab admin admin] PATH_INFO: `/v3/projects` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:21:57.520757 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c515efb-366d-4759-8d75-222d65b68bab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:21:57.522922 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c515efb-366d-4759-8d75-222d65b68bab admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:21:57.634865 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1346/2681] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 09:21:57 2022] POST /identity/v3/projects => generated 377 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 09:21:57.639084 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a103b9ae-d33a-40b7-97ce-e328c048f267 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:21:57.649018 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a103b9ae-d33a-40b7-97ce-e328c048f267 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:21:57.706375 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0128b8b3-8c67-4165-b9a1-c5f3c3802839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:21:57.706977 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0128b8b3-8c67-4165-b9a1-c5f3c3802839 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:21:57.707255 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0128b8b3-8c67-4165-b9a1-c5f3c3802839 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:21:57.707524 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0128b8b3-8c67-4165-b9a1-c5f3c3802839 admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:21:57.708163 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0128b8b3-8c67-4165-b9a1-c5f3c3802839 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:21:57.710635 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0128b8b3-8c67-4165-b9a1-c5f3c3802839 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:21:57.781538 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1336/2682] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 09:21:57 2022] POST /identity/v3/users => generated 289 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 09:21:57.785722 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c515efb-366d-4759-8d75-222d65b68bab admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:21:57.797172 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c515efb-366d-4759-8d75-222d65b68bab admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:21:57.801688 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86af4788-2359-41ab-a25f-a88230442674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:21:57.802282 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86af4788-2359-41ab-a25f-a88230442674 admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:21:57.802555 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86af4788-2359-41ab-a25f-a88230442674 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:21:57.802873 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86af4788-2359-41ab-a25f-a88230442674 admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:21:57.804647 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86af4788-2359-41ab-a25f-a88230442674 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:21:57.806268 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86af4788-2359-41ab-a25f-a88230442674 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:21:57.815893 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1347/2683] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 09:21:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 09:21:57.819859 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0128b8b3-8c67-4165-b9a1-c5f3c3802839 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:21:57.829979 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0128b8b3-8c67-4165-b9a1-c5f3c3802839 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:21:57.834806 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa056683-24e8-4ac1-b347-47362c8d841b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:21:57.835389 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa056683-24e8-4ac1-b347-47362c8d841b admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:21:57.835694 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa056683-24e8-4ac1-b347-47362c8d841b admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:21:57.835963 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa056683-24e8-4ac1-b347-47362c8d841b admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:21:57.836600 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa056683-24e8-4ac1-b347-47362c8d841b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:21:57.838309 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa056683-24e8-4ac1-b347-47362c8d841b admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:21:57.848627 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1337/2684] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 09:21:57 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 09:21:57.853321 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86af4788-2359-41ab-a25f-a88230442674 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:21:57.863003 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86af4788-2359-41ab-a25f-a88230442674 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:21:57.868663 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3061075b-fc96-4d85-b7a3-202853e4ad23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:21:57.869388 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3061075b-fc96-4d85-b7a3-202853e4ad23 admin admin] REQUEST_METHOD: `PUT` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:21:57.869664 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3061075b-fc96-4d85-b7a3-202853e4ad23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:21:57.869932 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3061075b-fc96-4d85-b7a3-202853e4ad23 admin admin] PATH_INFO: `/v3/projects/553d9ba584d34505931b879a2fc81c29/users/aeb39992700242d7b4511f693b9354d7/roles/44b64db484f24a46bf0b7ced46b519bf` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:21:57.884382 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3061075b-fc96-4d85-b7a3-202853e4ad23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=553d9ba584d34505931b879a2fc81c29, user_id=aeb39992700242d7b4511f693b9354d7, role_id=44b64db484f24a46bf0b7ced46b519bf)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:21:57.887039 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3061075b-fc96-4d85-b7a3-202853e4ad23 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:21:57.889168 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:21:57.947316 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1348/2685] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 09:21:57 2022] PUT /identity/v3/projects/553d9ba584d34505931b879a2fc81c29/users/aeb39992700242d7b4511f693b9354d7/roles/44b64db484f24a46bf0b7ced46b519bf => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:21:57.957614 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25824b23-d90b-45a2-85f0-d198a5b30635 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:21:57.957941 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25824b23-d90b-45a2-85f0-d198a5b30635 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:21:57.958220 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25824b23-d90b-45a2-85f0-d198a5b30635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:21:57.989648 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-25824b23-d90b-45a2-85f0-d198a5b30635 None None] MFA Rules not processed for user `aeb39992700242d7b4511f693b9354d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:21:57.991476 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-25824b23-d90b-45a2-85f0-d198a5b30635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:21:58.023196 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:21:58.057015 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1338/2686] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 09:21:57 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:21:58.067391 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3061075b-fc96-4d85-b7a3-202853e4ad23 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:21:58.069966 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-3061075b-fc96-4d85-b7a3-202853e4ad23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:21:58.148383 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3061075b-fc96-4d85-b7a3-202853e4ad23 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:21:58.152270 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b294f43f-8af8-4833-a35a-89979b1d9eee service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3263af99fc2f48e79fd1e5b8a1e555a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:21:58.152995 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b294f43f-8af8-4833-a35a-89979b1d9eee service glance] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:21:58.153278 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b294f43f-8af8-4833-a35a-89979b1d9eee service glance] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:21:58.153549 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b294f43f-8af8-4833-a35a-89979b1d9eee service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:21:58.163570 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b294f43f-8af8-4833-a35a-89979b1d9eee service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:21:58.165807 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b294f43f-8af8-4833-a35a-89979b1d9eee service glance] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:21:58.174923 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1349/2687] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:21:58 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:22:08.159074 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=12 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:22:08.170648 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:INSERT=1,keystone:SELECT=37 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:22:36.327289 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25824b23-d90b-45a2-85f0-d198a5b30635 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:22:36.401138 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25824b23-d90b-45a2-85f0-d198a5b30635 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:22:36.408353 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da1de44-6e07-40d4-aaa2-290db455acc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:22:36.408353 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da1de44-6e07-40d4-aaa2-290db455acc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:22:36.408353 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da1de44-6e07-40d4-aaa2-290db455acc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:22:36.408353 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da1de44-6e07-40d4-aaa2-290db455acc2 admin admin] PATH_INFO: `/v3/users/aeb39992700242d7b4511f693b9354d7` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:22:36.408932 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da1de44-6e07-40d4-aaa2-290db455acc2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aeb39992700242d7b4511f693b9354d7)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:22:36.410521 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da1de44-6e07-40d4-aaa2-290db455acc2 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:22:36.575037 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-0da1de44-6e07-40d4-aaa2-290db455acc2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aeb39992700242d7b4511f693b9354d7'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:22:36.588442 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-0da1de44-6e07-40d4-aaa2-290db455acc2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aeb39992700242d7b4511f693b9354d7'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:22:36.595561 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-0da1de44-6e07-40d4-aaa2-290db455acc2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aeb39992700242d7b4511f693b9354d7'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:22:36.596298 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-0da1de44-6e07-40d4-aaa2-290db455acc2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aeb39992700242d7b4511f693b9354d7'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:22:36.596900 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-0da1de44-6e07-40d4-aaa2-290db455acc2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aeb39992700242d7b4511f693b9354d7'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:22:36.641036 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1339/2688] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 09:22:36 2022] DELETE /identity/v3/users/aeb39992700242d7b4511f693b9354d7 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:22:36.645903 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940cf936-c68a-4c49-a0f5-211a678c7d50 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:22:36.646372 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940cf936-c68a-4c49-a0f5-211a678c7d50 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:22:36.646607 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940cf936-c68a-4c49-a0f5-211a678c7d50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:22:36.669098 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-940cf936-c68a-4c49-a0f5-211a678c7d50 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:22:36.670288 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-940cf936-c68a-4c49-a0f5-211a678c7d50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:22:36.731106 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1350/2689] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 09:22:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:22:36.741875 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da1de44-6e07-40d4-aaa2-290db455acc2 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:22:36.743316 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-0da1de44-6e07-40d4-aaa2-290db455acc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:22:36.804638 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da1de44-6e07-40d4-aaa2-290db455acc2 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:22:36.808371 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691f47de-186d-48e1-9e63-bdfb922578ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:22:36.808996 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691f47de-186d-48e1-9e63-bdfb922578ed service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:22:36.809274 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691f47de-186d-48e1-9e63-bdfb922578ed service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:22:36.809546 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691f47de-186d-48e1-9e63-bdfb922578ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:22:36.819544 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691f47de-186d-48e1-9e63-bdfb922578ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:22:36.820960 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691f47de-186d-48e1-9e63-bdfb922578ed service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:22:36.830999 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1340/2690] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:22:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:22:36.946889 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940cf936-c68a-4c49-a0f5-211a678c7d50 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:22:36.948325 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-940cf936-c68a-4c49-a0f5-211a678c7d50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:22:36.960675 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940cf936-c68a-4c49-a0f5-211a678c7d50 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:22:36.965215 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656b302d-9a71-4088-a6a1-8334cfe2f6f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:22:36.965832 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656b302d-9a71-4088-a6a1-8334cfe2f6f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:22:36.966109 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656b302d-9a71-4088-a6a1-8334cfe2f6f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:22:36.966379 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656b302d-9a71-4088-a6a1-8334cfe2f6f9 admin admin] PATH_INFO: `/v3/users/2a6365dc52f0402db08b151014dbb22e` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:22:36.967476 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656b302d-9a71-4088-a6a1-8334cfe2f6f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a6365dc52f0402db08b151014dbb22e)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:22:36.970304 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656b302d-9a71-4088-a6a1-8334cfe2f6f9 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:22:37.151832 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-656b302d-9a71-4088-a6a1-8334cfe2f6f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a6365dc52f0402db08b151014dbb22e'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:22:37.164059 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-656b302d-9a71-4088-a6a1-8334cfe2f6f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a6365dc52f0402db08b151014dbb22e'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:22:37.177881 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-656b302d-9a71-4088-a6a1-8334cfe2f6f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a6365dc52f0402db08b151014dbb22e'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:22:37.202310 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691f47de-186d-48e1-9e63-bdfb922578ed service neutron] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:22:37.204269 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-691f47de-186d-48e1-9e63-bdfb922578ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:22:37.217105 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691f47de-186d-48e1-9e63-bdfb922578ed service neutron] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:22:37.222066 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d862d216-65a8-41a6-9b8f-c0dcfdd512a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:22:37.222338 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d862d216-65a8-41a6-9b8f-c0dcfdd512a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:22:37.222621 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d862d216-65a8-41a6-9b8f-c0dcfdd512a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:22:37.222889 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d862d216-65a8-41a6-9b8f-c0dcfdd512a6 admin admin] PATH_INFO: `/v3/projects/553d9ba584d34505931b879a2fc81c29` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:22:37.224006 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d862d216-65a8-41a6-9b8f-c0dcfdd512a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=553d9ba584d34505931b879a2fc81c29)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:22:37.225974 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d862d216-65a8-41a6-9b8f-c0dcfdd512a6 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:22:37.243227 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:22:37.269667 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-656b302d-9a71-4088-a6a1-8334cfe2f6f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a6365dc52f0402db08b151014dbb22e'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:22:37.270209 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-656b302d-9a71-4088-a6a1-8334cfe2f6f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a6365dc52f0402db08b151014dbb22e'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:22:37.276485 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1351/2691] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 09:22:36 2022] DELETE /identity/v3/users/2a6365dc52f0402db08b151014dbb22e => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:22:37.280900 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4396446c-2595-4aab-9dd1-7ebdff870be3 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:22:37.281202 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4396446c-2595-4aab-9dd1-7ebdff870be3 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:22:37.281495 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4396446c-2595-4aab-9dd1-7ebdff870be3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:22:37.300024 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-4396446c-2595-4aab-9dd1-7ebdff870be3 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:22:37.301236 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-4396446c-2595-4aab-9dd1-7ebdff870be3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:22:37.359095 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1352/2692] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 09:22:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 09:22:37.360649 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4396446c-2595-4aab-9dd1-7ebdff870be3 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:22:37.366046 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-4396446c-2595-4aab-9dd1-7ebdff870be3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:22:37.373215 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-d862d216-65a8-41a6-9b8f-c0dcfdd512a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '553d9ba584d34505931b879a2fc81c29', 'request_id': 'req-d862d216-65a8-41a6-9b8f-c0dcfdd512a6'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:22:37.381122 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:22:37.423375 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4396446c-2595-4aab-9dd1-7ebdff870be3 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:22:37.428979 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-d862d216-65a8-41a6-9b8f-c0dcfdd512a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '553d9ba584d34505931b879a2fc81c29', 'request_id': 'req-d862d216-65a8-41a6-9b8f-c0dcfdd512a6'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:22:37.435267 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-d862d216-65a8-41a6-9b8f-c0dcfdd512a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '553d9ba584d34505931b879a2fc81c29', 'request_id': 'req-d862d216-65a8-41a6-9b8f-c0dcfdd512a6'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:22:37.447324 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-d862d216-65a8-41a6-9b8f-c0dcfdd512a6 admin admin] The token cache is being invalidate because project 553d9ba584d34505931b879a2fc81c29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 09:22:37.447618 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-d862d216-65a8-41a6-9b8f-c0dcfdd512a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:22:37.448250 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-d862d216-65a8-41a6-9b8f-c0dcfdd512a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:22:37.449776 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1341/2693] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 09:22:37 2022] DELETE /identity/v3/projects/553d9ba584d34505931b879a2fc81c29 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 09:22:37.450540 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d862d216-65a8-41a6-9b8f-c0dcfdd512a6 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:22:37.455542 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-d862d216-65a8-41a6-9b8f-c0dcfdd512a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:22:37.472227 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159a984c-d094-4f09-bc65-8205d47f996b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ba74e28ced4cbba857f54222dfbe41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:22:37.472826 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159a984c-d094-4f09-bc65-8205d47f996b service cinder] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:22:37.473130 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159a984c-d094-4f09-bc65-8205d47f996b service cinder] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:22:37.473399 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159a984c-d094-4f09-bc65-8205d47f996b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:22:37.475168 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-159a984c-d094-4f09-bc65-8205d47f996b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:22:37.512506 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d862d216-65a8-41a6-9b8f-c0dcfdd512a6 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:22:37.517045 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d31c097-478a-4fa6-928a-f440a13be33d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:22:37.517649 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d31c097-478a-4fa6-928a-f440a13be33d service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:22:37.517928 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d31c097-478a-4fa6-928a-f440a13be33d service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:22:37.518203 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d31c097-478a-4fa6-928a-f440a13be33d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:22:37.521419 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-5d31c097-478a-4fa6-928a-f440a13be33d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:22:37.533920 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159a984c-d094-4f09-bc65-8205d47f996b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:22:37.535571 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159a984c-d094-4f09-bc65-8205d47f996b service cinder] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:22:37.545841 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1353/2694] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:22:37 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:22:37.550070 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fa3c4d-4a74-4d89-bb05-84e4966e5b79 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:22:37.550361 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fa3c4d-4a74-4d89-bb05-84e4966e5b79 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:22:37.550638 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fa3c4d-4a74-4d89-bb05-84e4966e5b79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:22:37.574226 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-f0fa3c4d-4a74-4d89-bb05-84e4966e5b79 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:22:37.575158 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-f0fa3c4d-4a74-4d89-bb05-84e4966e5b79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:22:37.587537 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d31c097-478a-4fa6-928a-f440a13be33d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:22:37.589071 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d31c097-478a-4fa6-928a-f440a13be33d service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:22:37.606857 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1342/2695] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:22:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:22:37.644151 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1354/2696] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 09:22:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:22:37.651917 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f23c87-4926-44cc-9249-51f6e5c22e5d None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:22:37.652242 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f23c87-4926-44cc-9249-51f6e5c22e5d None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:22:37.652510 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f23c87-4926-44cc-9249-51f6e5c22e5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:22:37.683172 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-b7f23c87-4926-44cc-9249-51f6e5c22e5d None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:22:37.684466 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-b7f23c87-4926-44cc-9249-51f6e5c22e5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:22:37.693656 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1343/2697] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 09:22:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:22:37.698018 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fa3c4d-4a74-4d89-bb05-84e4966e5b79 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:22:37.708660 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fa3c4d-4a74-4d89-bb05-84e4966e5b79 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:22:37.713487 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52871eb7-5331-411e-9ff0-abc3cf905c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:22:37.714107 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52871eb7-5331-411e-9ff0-abc3cf905c35 admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:22:37.714365 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52871eb7-5331-411e-9ff0-abc3cf905c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:22:37.714632 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52871eb7-5331-411e-9ff0-abc3cf905c35 admin admin] PATH_INFO: `/v3/domains` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:22:37.715276 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52871eb7-5331-411e-9ff0-abc3cf905c35 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:22:37.715575 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52871eb7-5331-411e-9ff0-abc3cf905c35 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:22:37.717230 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52871eb7-5331-411e-9ff0-abc3cf905c35 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:22:37.729299 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1355/2698] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Aug 10 09:22:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:22:37.733686 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f23c87-4926-44cc-9249-51f6e5c22e5d None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:22:37.743332 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f23c87-4926-44cc-9249-51f6e5c22e5d None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:22:37.748025 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2b0f8a-03a9-4224-8f63-4e82c70b59fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:22:37.749900 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2b0f8a-03a9-4224-8f63-4e82c70b59fc admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:22:37.749900 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2b0f8a-03a9-4224-8f63-4e82c70b59fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:22:37.749900 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2b0f8a-03a9-4224-8f63-4e82c70b59fc admin admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:22:37.749900 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2b0f8a-03a9-4224-8f63-4e82c70b59fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:22:37.752270 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2b0f8a-03a9-4224-8f63-4e82c70b59fc admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:22:37.848770 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1344/2699] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 09:22:37 2022] POST /identity/v3/projects => generated 367 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 09:22:37.852997 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52871eb7-5331-411e-9ff0-abc3cf905c35 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:22:37.864010 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52871eb7-5331-411e-9ff0-abc3cf905c35 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:22:37.867676 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:22:37.922518 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec53cf2-f4f9-42e0-a004-25e2d97726d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:22:37.923158 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec53cf2-f4f9-42e0-a004-25e2d97726d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:22:37.923439 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec53cf2-f4f9-42e0-a004-25e2d97726d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:22:37.923711 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec53cf2-f4f9-42e0-a004-25e2d97726d4 admin admin] PATH_INFO: `/v3/users` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:22:37.924331 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec53cf2-f4f9-42e0-a004-25e2d97726d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:22:37.926319 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec53cf2-f4f9-42e0-a004-25e2d97726d4 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:22:37.974107 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2b0f8a-03a9-4224-8f63-4e82c70b59fc admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:22:37.978066 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-7e2b0f8a-03a9-4224-8f63-4e82c70b59fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:22:37.991458 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2b0f8a-03a9-4224-8f63-4e82c70b59fc admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:22:37.996719 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a992dd8-f835-4eb5-9214-a86d0a4a4e2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:22:38.001584 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a992dd8-f835-4eb5-9214-a86d0a4a4e2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:22:38.001584 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a992dd8-f835-4eb5-9214-a86d0a4a4e2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:22:38.001584 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a992dd8-f835-4eb5-9214-a86d0a4a4e2d admin admin] PATH_INFO: `/v3/projects/11ac43eb3ba24931abfac99a1f1eb67c` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:22:38.002226 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1356/2700] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 09:22:37 2022] POST /identity/v3/users => generated 284 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 09:22:38.002329 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec53cf2-f4f9-42e0-a004-25e2d97726d4 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:22:38.002699 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a992dd8-f835-4eb5-9214-a86d0a4a4e2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=11ac43eb3ba24931abfac99a1f1eb67c)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:22:38.005492 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a992dd8-f835-4eb5-9214-a86d0a4a4e2d admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:22:38.012881 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec53cf2-f4f9-42e0-a004-25e2d97726d4 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:22:38.018628 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5974e9-ac65-4e40-909d-f930571950e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:22:38.019206 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5974e9-ac65-4e40-909d-f930571950e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:22:38.019479 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5974e9-ac65-4e40-909d-f930571950e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:22:38.019748 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5974e9-ac65-4e40-909d-f930571950e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:22:38.020245 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:22:38.020369 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5974e9-ac65-4e40-909d-f930571950e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:22:38.022441 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5974e9-ac65-4e40-909d-f930571950e5 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:22:38.033616 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1357/2701] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 09:22:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 09:22:38.037602 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5974e9-ac65-4e40-909d-f930571950e5 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:22:38.047999 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5974e9-ac65-4e40-909d-f930571950e5 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:22:38.052676 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1d6e39-ee2e-41df-ae68-d07b5372fc79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:22:38.053285 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1d6e39-ee2e-41df-ae68-d07b5372fc79 admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:22:38.053563 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1d6e39-ee2e-41df-ae68-d07b5372fc79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:22:38.053828 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1d6e39-ee2e-41df-ae68-d07b5372fc79 admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:22:38.054481 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1d6e39-ee2e-41df-ae68-d07b5372fc79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:22:38.056038 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1d6e39-ee2e-41df-ae68-d07b5372fc79 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:22:38.066762 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1358/2702] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 09:22:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 09:22:38.070811 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1d6e39-ee2e-41df-ae68-d07b5372fc79 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:22:38.081974 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1d6e39-ee2e-41df-ae68-d07b5372fc79 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:22:38.086823 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9470063-bbef-4a48-9785-7e8833581694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:22:38.087459 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9470063-bbef-4a48-9785-7e8833581694 admin admin] REQUEST_METHOD: `PUT` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:22:38.087734 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9470063-bbef-4a48-9785-7e8833581694 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:22:38.088004 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9470063-bbef-4a48-9785-7e8833581694 admin admin] PATH_INFO: `/v3/projects/e9f8c00c05c745358fb8f9e9cf6bc9da/users/8409645cd3c14e83a305654ba9b18f88/roles/44b64db484f24a46bf0b7ced46b519bf` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:22:38.102202 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9470063-bbef-4a48-9785-7e8833581694 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9f8c00c05c745358fb8f9e9cf6bc9da, user_id=8409645cd3c14e83a305654ba9b18f88, role_id=44b64db484f24a46bf0b7ced46b519bf)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:22:38.104830 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9470063-bbef-4a48-9785-7e8833581694 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:22:38.121592 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-4a992dd8-f835-4eb5-9214-a86d0a4a4e2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11ac43eb3ba24931abfac99a1f1eb67c', 'request_id': 'req-4a992dd8-f835-4eb5-9214-a86d0a4a4e2d'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:22:38.190436 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1359/2703] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 09:22:38 2022] PUT /identity/v3/projects/e9f8c00c05c745358fb8f9e9cf6bc9da/users/8409645cd3c14e83a305654ba9b18f88/roles/44b64db484f24a46bf0b7ced46b519bf => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 09:22:38.200539 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8f1601-09b1-4621-b4b8-2ca8af526d6f None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:22:38.200859 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8f1601-09b1-4621-b4b8-2ca8af526d6f None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:22:38.201162 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8f1601-09b1-4621-b4b8-2ca8af526d6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:22:38.207460 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-4a992dd8-f835-4eb5-9214-a86d0a4a4e2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11ac43eb3ba24931abfac99a1f1eb67c', 'request_id': 'req-4a992dd8-f835-4eb5-9214-a86d0a4a4e2d'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:22:38.213651 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-4a992dd8-f835-4eb5-9214-a86d0a4a4e2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11ac43eb3ba24931abfac99a1f1eb67c', 'request_id': 'req-4a992dd8-f835-4eb5-9214-a86d0a4a4e2d'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:22:38.226064 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-4a992dd8-f835-4eb5-9214-a86d0a4a4e2d admin admin] The token cache is being invalidate because project 11ac43eb3ba24931abfac99a1f1eb67c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 09:22:38.226344 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-4a992dd8-f835-4eb5-9214-a86d0a4a4e2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:22:38.227001 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-4a992dd8-f835-4eb5-9214-a86d0a4a4e2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:22:38.228578 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1345/2704] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 09:22:37 2022] DELETE /identity/v3/projects/11ac43eb3ba24931abfac99a1f1eb67c => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 09:22:38.229986 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-fc8f1601-09b1-4621-b4b8-2ca8af526d6f None None] MFA Rules not processed for user `8409645cd3c14e83a305654ba9b18f88`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:22:38.231224 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-fc8f1601-09b1-4621-b4b8-2ca8af526d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:22:38.232961 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a992dd8-f835-4eb5-9214-a86d0a4a4e2d admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:22:38.235092 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-4a992dd8-f835-4eb5-9214-a86d0a4a4e2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:22:38.301287 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a992dd8-f835-4eb5-9214-a86d0a4a4e2d admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:22:38.301473 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1360/2705] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 09:22:38 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 09:22:38.313610 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8f1601-09b1-4621-b4b8-2ca8af526d6f None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:22:38.314919 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-fc8f1601-09b1-4621-b4b8-2ca8af526d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:22:38.356194 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:22:38.358992 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a668a7-3952-493f-955a-f1088c2f4132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:22:38.360210 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a668a7-3952-493f-955a-f1088c2f4132 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:22:38.360507 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a668a7-3952-493f-955a-f1088c2f4132 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:22:38.360778 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a668a7-3952-493f-955a-f1088c2f4132 admin admin] PATH_INFO: `/v3/users/a01c5fd94e474195862db8e73d88deea` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:22:38.362043 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a668a7-3952-493f-955a-f1088c2f4132 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a01c5fd94e474195862db8e73d88deea)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:22:38.363955 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a668a7-3952-493f-955a-f1088c2f4132 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:22:38.377773 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8f1601-09b1-4621-b4b8-2ca8af526d6f None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:22:38.382666 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02891d4e-09d1-4531-8243-a5bfd434b1c9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3263af99fc2f48e79fd1e5b8a1e555a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:22:38.383474 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02891d4e-09d1-4531-8243-a5bfd434b1c9 service glance] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:22:38.383917 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02891d4e-09d1-4531-8243-a5bfd434b1c9 service glance] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:22:38.384450 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02891d4e-09d1-4531-8243-a5bfd434b1c9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:22:38.393675 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02891d4e-09d1-4531-8243-a5bfd434b1c9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:22:38.395136 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02891d4e-09d1-4531-8243-a5bfd434b1c9 service glance] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:22:38.404071 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1361/2706] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:22:38 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 09:22:38.517309 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-47a668a7-3952-493f-955a-f1088c2f4132 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a01c5fd94e474195862db8e73d88deea'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:22:38.519249 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:22:38.536156 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-47a668a7-3952-493f-955a-f1088c2f4132 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a01c5fd94e474195862db8e73d88deea'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:22:38.544780 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-47a668a7-3952-493f-955a-f1088c2f4132 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a01c5fd94e474195862db8e73d88deea'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:22:38.545611 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-47a668a7-3952-493f-955a-f1088c2f4132 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a01c5fd94e474195862db8e73d88deea'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:22:38.546245 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-47a668a7-3952-493f-955a-f1088c2f4132 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a01c5fd94e474195862db8e73d88deea'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:22:38.606269 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1346/2707] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 09:22:38 2022] DELETE /identity/v3/users/a01c5fd94e474195862db8e73d88deea => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 09:22:39.028799 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02891d4e-09d1-4531-8243-a5bfd434b1c9 service glance] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:22:39.045582 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02891d4e-09d1-4531-8243-a5bfd434b1c9 service glance] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:22:39.109479 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f81bc0c-f717-41c6-b470-7d1b0e67f3b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:22:39.110339 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f81bc0c-f717-41c6-b470-7d1b0e67f3b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:22:39.110616 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f81bc0c-f717-41c6-b470-7d1b0e67f3b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:22:39.110883 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f81bc0c-f717-41c6-b470-7d1b0e67f3b1 admin admin] PATH_INFO: `/v3/projects/14a7579e39034285b1837874e4df7123` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:22:39.112136 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f81bc0c-f717-41c6-b470-7d1b0e67f3b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=14a7579e39034285b1837874e4df7123)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:22:39.114861 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f81bc0c-f717-41c6-b470-7d1b0e67f3b1 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:22:39.241541 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-6f81bc0c-f717-41c6-b470-7d1b0e67f3b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14a7579e39034285b1837874e4df7123', 'request_id': 'req-6f81bc0c-f717-41c6-b470-7d1b0e67f3b1'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:22:39.246993 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-6f81bc0c-f717-41c6-b470-7d1b0e67f3b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14a7579e39034285b1837874e4df7123', 'request_id': 'req-6f81bc0c-f717-41c6-b470-7d1b0e67f3b1'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:22:39.338150 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-6f81bc0c-f717-41c6-b470-7d1b0e67f3b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14a7579e39034285b1837874e4df7123', 'request_id': 'req-6f81bc0c-f717-41c6-b470-7d1b0e67f3b1'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:22:39.350431 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-6f81bc0c-f717-41c6-b470-7d1b0e67f3b1 admin admin] The token cache is being invalidate because project 14a7579e39034285b1837874e4df7123 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 09:22:39.350716 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-6f81bc0c-f717-41c6-b470-7d1b0e67f3b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:22:39.351346 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-6f81bc0c-f717-41c6-b470-7d1b0e67f3b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:22:39.352829 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1362/2708] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 09:22:39 2022] DELETE /identity/v3/projects/14a7579e39034285b1837874e4df7123 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:22:39.357802 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaf5023-b351-492c-874c-13acfc7d1b47 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:22:39.358117 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaf5023-b351-492c-874c-13acfc7d1b47 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:22:39.358396 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaf5023-b351-492c-874c-13acfc7d1b47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:22:39.378326 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-0aaf5023-b351-492c-874c-13acfc7d1b47 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:22:39.379519 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-0aaf5023-b351-492c-874c-13acfc7d1b47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:22:39.439978 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1347/2709] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 09:22:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:22:39.448055 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd87412e-0251-4a77-bf3e-81aa30a719fe None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:22:39.448384 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd87412e-0251-4a77-bf3e-81aa30a719fe None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:22:39.448666 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd87412e-0251-4a77-bf3e-81aa30a719fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:22:39.468097 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-bd87412e-0251-4a77-bf3e-81aa30a719fe None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:22:39.469517 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-bd87412e-0251-4a77-bf3e-81aa30a719fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:22:39.477212 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1363/2710] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 09:22:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:22:39.481273 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaf5023-b351-492c-874c-13acfc7d1b47 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:22:39.490751 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaf5023-b351-492c-874c-13acfc7d1b47 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:22:39.495234 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4398cb95-2685-4b7d-9405-0ca5e69b8494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:22:39.495821 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4398cb95-2685-4b7d-9405-0ca5e69b8494 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:22:39.496100 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4398cb95-2685-4b7d-9405-0ca5e69b8494 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:22:39.496367 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4398cb95-2685-4b7d-9405-0ca5e69b8494 admin admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:22:39.497037 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4398cb95-2685-4b7d-9405-0ca5e69b8494 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:22:39.497340 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4398cb95-2685-4b7d-9405-0ca5e69b8494 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:22:39.498911 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4398cb95-2685-4b7d-9405-0ca5e69b8494 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:22:39.511656 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1348/2711] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Aug 10 09:22:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:22:39.516548 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd87412e-0251-4a77-bf3e-81aa30a719fe None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:22:39.526015 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd87412e-0251-4a77-bf3e-81aa30a719fe None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:22:39.530674 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bfcbf9-baf4-4725-bda9-6a9f611217e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:22:39.531266 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bfcbf9-baf4-4725-bda9-6a9f611217e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:22:39.531715 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bfcbf9-baf4-4725-bda9-6a9f611217e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:22:39.532305 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bfcbf9-baf4-4725-bda9-6a9f611217e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:22:39.532780 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bfcbf9-baf4-4725-bda9-6a9f611217e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:22:39.534728 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bfcbf9-baf4-4725-bda9-6a9f611217e7 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:22:39.614119 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1364/2712] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 09:22:39 2022] POST /identity/v3/projects => generated 409 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 09:22:39.618258 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4398cb95-2685-4b7d-9405-0ca5e69b8494 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:22:39.629185 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4398cb95-2685-4b7d-9405-0ca5e69b8494 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:22:39.688712 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213d9953-58b4-4bed-9d24-532f1982bdd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:22:39.689669 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213d9953-58b4-4bed-9d24-532f1982bdd4 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:22:39.689669 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213d9953-58b4-4bed-9d24-532f1982bdd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:22:39.689875 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213d9953-58b4-4bed-9d24-532f1982bdd4 admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:22:39.690512 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213d9953-58b4-4bed-9d24-532f1982bdd4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:22:39.692461 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213d9953-58b4-4bed-9d24-532f1982bdd4 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:22:39.768106 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1349/2713] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 09:22:39 2022] POST /identity/v3/users => generated 305 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 09:22:39.772363 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bfcbf9-baf4-4725-bda9-6a9f611217e7 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:22:39.783024 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bfcbf9-baf4-4725-bda9-6a9f611217e7 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:22:39.787675 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a7adc7-43cc-42d3-be22-8a3b480f4d55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:22:39.788266 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a7adc7-43cc-42d3-be22-8a3b480f4d55 admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:22:39.788541 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a7adc7-43cc-42d3-be22-8a3b480f4d55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:22:39.788809 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a7adc7-43cc-42d3-be22-8a3b480f4d55 admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:22:39.789727 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a7adc7-43cc-42d3-be22-8a3b480f4d55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:22:39.793011 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a7adc7-43cc-42d3-be22-8a3b480f4d55 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:22:39.803696 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1365/2714] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 09:22:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 09:22:39.808048 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213d9953-58b4-4bed-9d24-532f1982bdd4 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:22:39.818727 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213d9953-58b4-4bed-9d24-532f1982bdd4 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:22:39.823223 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476d955c-c4c8-46bc-b0bd-79ee7684954d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:22:39.823802 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476d955c-c4c8-46bc-b0bd-79ee7684954d admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:22:39.824080 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476d955c-c4c8-46bc-b0bd-79ee7684954d admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:22:39.824348 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476d955c-c4c8-46bc-b0bd-79ee7684954d admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:22:39.825010 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476d955c-c4c8-46bc-b0bd-79ee7684954d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:22:39.826584 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476d955c-c4c8-46bc-b0bd-79ee7684954d admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:22:39.836930 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1350/2715] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 09:22:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 09:22:39.841037 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a7adc7-43cc-42d3-be22-8a3b480f4d55 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:22:39.851492 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a7adc7-43cc-42d3-be22-8a3b480f4d55 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:22:39.856439 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44450fe4-601d-4a14-8900-333047b84b87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:22:39.857152 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44450fe4-601d-4a14-8900-333047b84b87 admin admin] REQUEST_METHOD: `PUT` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:22:39.857435 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44450fe4-601d-4a14-8900-333047b84b87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:22:39.857710 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44450fe4-601d-4a14-8900-333047b84b87 admin admin] PATH_INFO: `/v3/projects/c6ea5279a42b4ff0873da5748bffbd50/users/e3cc686c6c074a51a78f3bdee8d8b13c/roles/44b64db484f24a46bf0b7ced46b519bf` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:22:39.873273 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44450fe4-601d-4a14-8900-333047b84b87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6ea5279a42b4ff0873da5748bffbd50, user_id=e3cc686c6c074a51a78f3bdee8d8b13c, role_id=44b64db484f24a46bf0b7ced46b519bf)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:22:39.876642 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44450fe4-601d-4a14-8900-333047b84b87 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:22:39.945240 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1366/2716] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 09:22:39 2022] PUT /identity/v3/projects/c6ea5279a42b4ff0873da5748bffbd50/users/e3cc686c6c074a51a78f3bdee8d8b13c/roles/44b64db484f24a46bf0b7ced46b519bf => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:22:39.956473 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bd1f0a-44ab-40dc-a205-8bc642db536d None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:22:39.956756 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bd1f0a-44ab-40dc-a205-8bc642db536d None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:22:39.957053 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bd1f0a-44ab-40dc-a205-8bc642db536d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:22:39.985487 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-75bd1f0a-44ab-40dc-a205-8bc642db536d None None] MFA Rules not processed for user `e3cc686c6c074a51a78f3bdee8d8b13c`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:22:39.986801 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-75bd1f0a-44ab-40dc-a205-8bc642db536d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:22:40.006880 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:22:40.052969 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1351/2717] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 09:22:39 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:22:40.057491 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44450fe4-601d-4a14-8900-333047b84b87 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:22:40.061924 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:22:40.122546 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44450fe4-601d-4a14-8900-333047b84b87 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:22:40.127534 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba82bb50-3049-4ee7-afe5-916ae669fcd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:22:40.128077 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba82bb50-3049-4ee7-afe5-916ae669fcd9 admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:22:40.128358 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba82bb50-3049-4ee7-afe5-916ae669fcd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:22:40.128630 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba82bb50-3049-4ee7-afe5-916ae669fcd9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:22:40.129387 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba82bb50-3049-4ee7-afe5-916ae669fcd9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:22:40.131377 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba82bb50-3049-4ee7-afe5-916ae669fcd9 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:22:40.214550 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1367/2718] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 09:22:40 2022] POST /identity/v3/projects => generated 409 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 09:22:40.221383 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bd1f0a-44ab-40dc-a205-8bc642db536d None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:22:40.232221 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bd1f0a-44ab-40dc-a205-8bc642db536d None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:22:40.287509 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90ec783-c7ca-4886-9b71-32a72b571043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:22:40.288214 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90ec783-c7ca-4886-9b71-32a72b571043 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:22:40.288562 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90ec783-c7ca-4886-9b71-32a72b571043 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:22:40.288839 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90ec783-c7ca-4886-9b71-32a72b571043 admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:22:40.289750 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90ec783-c7ca-4886-9b71-32a72b571043 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:22:40.291648 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90ec783-c7ca-4886-9b71-32a72b571043 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:22:40.372980 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1352/2719] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 09:22:40 2022] POST /identity/v3/users => generated 311 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 09:22:40.376907 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba82bb50-3049-4ee7-afe5-916ae669fcd9 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:22:40.395227 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba82bb50-3049-4ee7-afe5-916ae669fcd9 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:22:40.399979 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b353ed87-6796-4786-b8a6-53ec140bb28a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:22:40.400684 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b353ed87-6796-4786-b8a6-53ec140bb28a admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:22:40.400989 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b353ed87-6796-4786-b8a6-53ec140bb28a admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:22:40.401286 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b353ed87-6796-4786-b8a6-53ec140bb28a admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:22:40.402001 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b353ed87-6796-4786-b8a6-53ec140bb28a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:22:40.403596 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b353ed87-6796-4786-b8a6-53ec140bb28a admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:22:40.415804 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1368/2720] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 09:22:40 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 09:22:40.420041 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90ec783-c7ca-4886-9b71-32a72b571043 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:22:40.430594 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90ec783-c7ca-4886-9b71-32a72b571043 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:22:40.435180 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba1c3e8-0200-4a4b-89a3-c36e4ef2d190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:22:40.435817 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba1c3e8-0200-4a4b-89a3-c36e4ef2d190 admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:22:40.436093 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba1c3e8-0200-4a4b-89a3-c36e4ef2d190 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:22:40.436363 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba1c3e8-0200-4a4b-89a3-c36e4ef2d190 admin admin] PATH_INFO: `/v3/projects/356160df80514e23a13693e8f145b59a/users/d3312bfd38ae47a7b6b430383eb0f751/roles/ec3751692d9c4ecf81b4a71c936c46be` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:22:40.451221 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba1c3e8-0200-4a4b-89a3-c36e4ef2d190 admin admin] RBAC: Authorizing `identity:create_grant(project_id=356160df80514e23a13693e8f145b59a, user_id=d3312bfd38ae47a7b6b430383eb0f751, role_id=ec3751692d9c4ecf81b4a71c936c46be)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:22:40.454742 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba1c3e8-0200-4a4b-89a3-c36e4ef2d190 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:22:40.522139 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1353/2721] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 09:22:40 2022] PUT /identity/v3/projects/356160df80514e23a13693e8f145b59a/users/d3312bfd38ae47a7b6b430383eb0f751/roles/ec3751692d9c4ecf81b4a71c936c46be => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:22:40.526119 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b353ed87-6796-4786-b8a6-53ec140bb28a admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:22:40.536751 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b353ed87-6796-4786-b8a6-53ec140bb28a admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:22:40.600977 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06aa1e5a-1491-482a-bfbc-8dd195758c37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:22:40.601614 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aa1e5a-1491-482a-bfbc-8dd195758c37 admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:22:40.601915 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aa1e5a-1491-482a-bfbc-8dd195758c37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:22:40.602184 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aa1e5a-1491-482a-bfbc-8dd195758c37 admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:22:40.602807 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06aa1e5a-1491-482a-bfbc-8dd195758c37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:22:40.604389 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06aa1e5a-1491-482a-bfbc-8dd195758c37 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:22:40.614059 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1369/2722] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 09:22:40 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 09:22:40.618215 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba1c3e8-0200-4a4b-89a3-c36e4ef2d190 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:22:40.628270 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba1c3e8-0200-4a4b-89a3-c36e4ef2d190 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:22:40.632788 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4ad0bb-1107-459b-94d5-7515b8c4a69c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:22:40.633455 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4ad0bb-1107-459b-94d5-7515b8c4a69c admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:22:40.633730 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4ad0bb-1107-459b-94d5-7515b8c4a69c admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:22:40.633999 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4ad0bb-1107-459b-94d5-7515b8c4a69c admin admin] PATH_INFO: `/v3/projects/356160df80514e23a13693e8f145b59a/users/d3312bfd38ae47a7b6b430383eb0f751/roles/ec3751692d9c4ecf81b4a71c936c46be` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:22:40.635874 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4ad0bb-1107-459b-94d5-7515b8c4a69c admin admin] RBAC: Authorizing `identity:create_grant(project_id=356160df80514e23a13693e8f145b59a, user_id=d3312bfd38ae47a7b6b430383eb0f751, role_id=ec3751692d9c4ecf81b4a71c936c46be)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:22:40.638500 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4ad0bb-1107-459b-94d5-7515b8c4a69c admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:22:40.653494 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1354/2723] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 09:22:40 2022] PUT /identity/v3/projects/356160df80514e23a13693e8f145b59a/users/d3312bfd38ae47a7b6b430383eb0f751/roles/ec3751692d9c4ecf81b4a71c936c46be => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:22:40.662882 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495d3332-adf9-4528-807a-9ca6c6de11ff None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:22:40.663160 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495d3332-adf9-4528-807a-9ca6c6de11ff None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:22:40.663430 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495d3332-adf9-4528-807a-9ca6c6de11ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:22:40.691048 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-495d3332-adf9-4528-807a-9ca6c6de11ff None None] MFA Rules not processed for user `d3312bfd38ae47a7b6b430383eb0f751`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:22:40.692240 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-495d3332-adf9-4528-807a-9ca6c6de11ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:22:40.763149 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:22:40.770232 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1370/2724] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 09:22:40 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:22:40.780867 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4ad0bb-1107-459b-94d5-7515b8c4a69c admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:22:40.782853 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-7b4ad0bb-1107-459b-94d5-7515b8c4a69c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:22:40.843289 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4ad0bb-1107-459b-94d5-7515b8c4a69c admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:22:40.847143 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b26ff97-a230-4951-9179-b86da9df9e11 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ba74e28ced4cbba857f54222dfbe41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:22:40.847741 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b26ff97-a230-4951-9179-b86da9df9e11 service cinder] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:22:40.848016 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b26ff97-a230-4951-9179-b86da9df9e11 service cinder] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:22:40.848285 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b26ff97-a230-4951-9179-b86da9df9e11 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:22:40.857114 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b26ff97-a230-4951-9179-b86da9df9e11 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:22:40.858569 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b26ff97-a230-4951-9179-b86da9df9e11 service cinder] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:22:40.869281 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1355/2725] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:22:40 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:22:41.305475 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495d3332-adf9-4528-807a-9ca6c6de11ff None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:22:41.315778 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495d3332-adf9-4528-807a-9ca6c6de11ff None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:22:41.319320 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c351fc5-dc43-4dff-9a5d-3e7953b39c87 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ba74e28ced4cbba857f54222dfbe41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:22:41.319911 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c351fc5-dc43-4dff-9a5d-3e7953b39c87 service cinder] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:22:41.320189 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c351fc5-dc43-4dff-9a5d-3e7953b39c87 service cinder] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:22:41.320458 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c351fc5-dc43-4dff-9a5d-3e7953b39c87 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:22:41.378403 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c351fc5-dc43-4dff-9a5d-3e7953b39c87 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:22:41.379822 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c351fc5-dc43-4dff-9a5d-3e7953b39c87 service cinder] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:22:41.388727 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1371/2726] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:22:41 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:22:50.864612 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=87,keystone:INSERT=5 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:22:51.384685 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=29 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:22:56.302378 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b26ff97-a230-4951-9179-b86da9df9e11 service cinder] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:22:56.363803 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b26ff97-a230-4951-9179-b86da9df9e11 service cinder] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:22:56.368990 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eef6b6-5d8b-486e-8d6b-d22dbbd227fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:22:56.369604 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eef6b6-5d8b-486e-8d6b-d22dbbd227fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:22:56.369877 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eef6b6-5d8b-486e-8d6b-d22dbbd227fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:22:56.370142 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eef6b6-5d8b-486e-8d6b-d22dbbd227fc admin admin] PATH_INFO: `/v3/users/e3cc686c6c074a51a78f3bdee8d8b13c` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:22:56.371240 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44eef6b6-5d8b-486e-8d6b-d22dbbd227fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3cc686c6c074a51a78f3bdee8d8b13c)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:22:56.373165 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44eef6b6-5d8b-486e-8d6b-d22dbbd227fc admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:22:56.523215 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-44eef6b6-5d8b-486e-8d6b-d22dbbd227fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3cc686c6c074a51a78f3bdee8d8b13c'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:22:56.535013 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-44eef6b6-5d8b-486e-8d6b-d22dbbd227fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3cc686c6c074a51a78f3bdee8d8b13c'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:22:56.542784 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-44eef6b6-5d8b-486e-8d6b-d22dbbd227fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3cc686c6c074a51a78f3bdee8d8b13c'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:22:56.543489 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-44eef6b6-5d8b-486e-8d6b-d22dbbd227fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3cc686c6c074a51a78f3bdee8d8b13c'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:22:56.544093 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-44eef6b6-5d8b-486e-8d6b-d22dbbd227fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3cc686c6c074a51a78f3bdee8d8b13c'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:22:56.624101 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1356/2727] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 09:22:56 2022] DELETE /identity/v3/users/e3cc686c6c074a51a78f3bdee8d8b13c => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:22:56.628578 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ae13a5-d3d7-4f1c-831e-f4ceead1c6f0 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:22:56.630343 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ae13a5-d3d7-4f1c-831e-f4ceead1c6f0 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:22:56.630343 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ae13a5-d3d7-4f1c-831e-f4ceead1c6f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:22:56.648162 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-89ae13a5-d3d7-4f1c-831e-f4ceead1c6f0 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:22:56.649508 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-89ae13a5-d3d7-4f1c-831e-f4ceead1c6f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:22:56.707590 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1372/2728] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 09:22:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:22:56.718195 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eef6b6-5d8b-486e-8d6b-d22dbbd227fc admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:22:56.719568 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-44eef6b6-5d8b-486e-8d6b-d22dbbd227fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:22:56.786653 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eef6b6-5d8b-486e-8d6b-d22dbbd227fc admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:22:56.791824 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a186408-98ff-47d2-adc8-dba5fe5a3386 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:22:56.792423 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a186408-98ff-47d2-adc8-dba5fe5a3386 service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:22:56.792750 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a186408-98ff-47d2-adc8-dba5fe5a3386 service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:22:56.793078 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a186408-98ff-47d2-adc8-dba5fe5a3386 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:22:56.803548 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a186408-98ff-47d2-adc8-dba5fe5a3386 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:22:56.807235 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a186408-98ff-47d2-adc8-dba5fe5a3386 service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:22:56.819058 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1357/2729] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:22:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:22:57.500582 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ae13a5-d3d7-4f1c-831e-f4ceead1c6f0 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:22:57.502074 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-89ae13a5-d3d7-4f1c-831e-f4ceead1c6f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:22:57.516236 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ae13a5-d3d7-4f1c-831e-f4ceead1c6f0 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:22:57.521124 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbdb66f-f5ff-4c78-82c5-db75ca47d4d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:22:57.521748 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbdb66f-f5ff-4c78-82c5-db75ca47d4d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:22:57.522025 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbdb66f-f5ff-4c78-82c5-db75ca47d4d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:22:57.522290 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbdb66f-f5ff-4c78-82c5-db75ca47d4d4 admin admin] PATH_INFO: `/v3/projects/c6ea5279a42b4ff0873da5748bffbd50` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:22:57.524421 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbdb66f-f5ff-4c78-82c5-db75ca47d4d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6ea5279a42b4ff0873da5748bffbd50)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:22:57.527296 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbdb66f-f5ff-4c78-82c5-db75ca47d4d4 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:22:57.629864 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-cfbdb66f-f5ff-4c78-82c5-db75ca47d4d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6ea5279a42b4ff0873da5748bffbd50', 'request_id': 'req-cfbdb66f-f5ff-4c78-82c5-db75ca47d4d4'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:22:57.635396 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-cfbdb66f-f5ff-4c78-82c5-db75ca47d4d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6ea5279a42b4ff0873da5748bffbd50', 'request_id': 'req-cfbdb66f-f5ff-4c78-82c5-db75ca47d4d4'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:22:57.693310 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-cfbdb66f-f5ff-4c78-82c5-db75ca47d4d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6ea5279a42b4ff0873da5748bffbd50', 'request_id': 'req-cfbdb66f-f5ff-4c78-82c5-db75ca47d4d4'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:22:57.706155 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-cfbdb66f-f5ff-4c78-82c5-db75ca47d4d4 admin admin] The token cache is being invalidate because project c6ea5279a42b4ff0873da5748bffbd50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 09:22:57.706445 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-cfbdb66f-f5ff-4c78-82c5-db75ca47d4d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:22:57.707153 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-cfbdb66f-f5ff-4c78-82c5-db75ca47d4d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:22:57.708670 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1373/2730] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 09:22:57 2022] DELETE /identity/v3/projects/c6ea5279a42b4ff0873da5748bffbd50 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:22:57.713179 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a186408-98ff-47d2-adc8-dba5fe5a3386 service neutron] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:22:57.722815 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a186408-98ff-47d2-adc8-dba5fe5a3386 service neutron] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:22:57.732116 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:22:57.779795 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4e7c7d-7795-4f1c-a7e7-1fd1e2c39ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:22:57.780379 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4e7c7d-7795-4f1c-a7e7-1fd1e2c39ecc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:22:57.780660 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4e7c7d-7795-4f1c-a7e7-1fd1e2c39ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:22:57.780955 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4e7c7d-7795-4f1c-a7e7-1fd1e2c39ecc admin admin] PATH_INFO: `/v3/users/d3312bfd38ae47a7b6b430383eb0f751` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:22:57.782069 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4e7c7d-7795-4f1c-a7e7-1fd1e2c39ecc admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3312bfd38ae47a7b6b430383eb0f751)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:22:57.783964 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4e7c7d-7795-4f1c-a7e7-1fd1e2c39ecc admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:22:57.940621 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-7b4e7c7d-7795-4f1c-a7e7-1fd1e2c39ecc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3312bfd38ae47a7b6b430383eb0f751'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:22:57.957295 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-7b4e7c7d-7795-4f1c-a7e7-1fd1e2c39ecc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3312bfd38ae47a7b6b430383eb0f751'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:22:57.967797 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-7b4e7c7d-7795-4f1c-a7e7-1fd1e2c39ecc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3312bfd38ae47a7b6b430383eb0f751'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:22:57.968500 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-7b4e7c7d-7795-4f1c-a7e7-1fd1e2c39ecc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3312bfd38ae47a7b6b430383eb0f751'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:22:57.969127 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-7b4e7c7d-7795-4f1c-a7e7-1fd1e2c39ecc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3312bfd38ae47a7b6b430383eb0f751'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:22:58.030073 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1358/2731] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 09:22:57 2022] DELETE /identity/v3/users/d3312bfd38ae47a7b6b430383eb0f751 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:22:58.437869 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbdb66f-f5ff-4c78-82c5-db75ca47d4d4 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:22:58.439306 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-cfbdb66f-f5ff-4c78-82c5-db75ca47d4d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:22:58.452934 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbdb66f-f5ff-4c78-82c5-db75ca47d4d4 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:22:58.510265 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45681018-ab37-4158-aaaf-ae52dc90ea04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:22:58.510892 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45681018-ab37-4158-aaaf-ae52dc90ea04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:22:58.511170 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45681018-ab37-4158-aaaf-ae52dc90ea04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:22:58.511442 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45681018-ab37-4158-aaaf-ae52dc90ea04 admin admin] PATH_INFO: `/v3/projects/356160df80514e23a13693e8f145b59a` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:22:58.512555 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45681018-ab37-4158-aaaf-ae52dc90ea04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=356160df80514e23a13693e8f145b59a)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:22:58.514492 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45681018-ab37-4158-aaaf-ae52dc90ea04 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:22:58.586110 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:22:58.603509 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-45681018-ab37-4158-aaaf-ae52dc90ea04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '356160df80514e23a13693e8f145b59a', 'request_id': 'req-45681018-ab37-4158-aaaf-ae52dc90ea04'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:22:58.608947 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-45681018-ab37-4158-aaaf-ae52dc90ea04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '356160df80514e23a13693e8f145b59a', 'request_id': 'req-45681018-ab37-4158-aaaf-ae52dc90ea04'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:22:58.681001 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-45681018-ab37-4158-aaaf-ae52dc90ea04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '356160df80514e23a13693e8f145b59a', 'request_id': 'req-45681018-ab37-4158-aaaf-ae52dc90ea04'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:22:58.692081 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-45681018-ab37-4158-aaaf-ae52dc90ea04 admin admin] The token cache is being invalidate because project 356160df80514e23a13693e8f145b59a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 09:22:58.692367 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-45681018-ab37-4158-aaaf-ae52dc90ea04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:22:58.693031 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-45681018-ab37-4158-aaaf-ae52dc90ea04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:22:58.694501 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1374/2732] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 09:22:58 2022] DELETE /identity/v3/projects/356160df80514e23a13693e8f145b59a => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:22:58.699729 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82637637-ebd2-4832-870c-ac03e0d85dbd None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:22:58.700012 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82637637-ebd2-4832-870c-ac03e0d85dbd None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:22:58.700281 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82637637-ebd2-4832-870c-ac03e0d85dbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:22:58.719880 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-82637637-ebd2-4832-870c-ac03e0d85dbd None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:22:58.721129 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-82637637-ebd2-4832-870c-ac03e0d85dbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:22:58.782107 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1359/2733] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 09:22:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:22:58.791192 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847f08cb-bf34-4ece-82c4-72f302dd3fa5 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:22:58.791479 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847f08cb-bf34-4ece-82c4-72f302dd3fa5 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:22:58.791753 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847f08cb-bf34-4ece-82c4-72f302dd3fa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:22:58.810942 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-847f08cb-bf34-4ece-82c4-72f302dd3fa5 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:22:58.812134 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-847f08cb-bf34-4ece-82c4-72f302dd3fa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:22:58.820409 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1375/2734] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 09:22:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:22:58.824561 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82637637-ebd2-4832-870c-ac03e0d85dbd None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:22:58.834214 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82637637-ebd2-4832-870c-ac03e0d85dbd None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:22:58.839498 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f666492-6ce9-483d-a4d6-e713f2f2247b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:22:58.840027 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f666492-6ce9-483d-a4d6-e713f2f2247b admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:22:58.840314 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f666492-6ce9-483d-a4d6-e713f2f2247b admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:22:58.840586 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f666492-6ce9-483d-a4d6-e713f2f2247b admin admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:22:58.841273 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f666492-6ce9-483d-a4d6-e713f2f2247b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:22:58.841580 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f666492-6ce9-483d-a4d6-e713f2f2247b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:22:58.843161 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f666492-6ce9-483d-a4d6-e713f2f2247b admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:22:58.854571 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1360/2735] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Aug 10 09:22:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:22:58.858498 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847f08cb-bf34-4ece-82c4-72f302dd3fa5 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:22:58.867890 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847f08cb-bf34-4ece-82c4-72f302dd3fa5 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:22:58.872522 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8c9618-d18e-4820-a26f-216dcbc1a646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:22:58.873151 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8c9618-d18e-4820-a26f-216dcbc1a646 admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:22:58.873480 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8c9618-d18e-4820-a26f-216dcbc1a646 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:22:58.873746 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8c9618-d18e-4820-a26f-216dcbc1a646 admin admin] PATH_INFO: `/v3/projects` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:22:58.874401 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8c9618-d18e-4820-a26f-216dcbc1a646 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:22:58.876433 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8c9618-d18e-4820-a26f-216dcbc1a646 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:22:58.992002 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1376/2736] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 09:22:58 2022] POST /identity/v3/projects => generated 375 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 09:22:58.996345 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f666492-6ce9-483d-a4d6-e713f2f2247b admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:22:59.006443 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f666492-6ce9-483d-a4d6-e713f2f2247b admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:22:59.025143 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:22:59.064047 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc07d6fd-d23a-4a19-9462-954197f66aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:22:59.064649 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc07d6fd-d23a-4a19-9462-954197f66aeb admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:22:59.064938 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc07d6fd-d23a-4a19-9462-954197f66aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:22:59.065225 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc07d6fd-d23a-4a19-9462-954197f66aeb admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:22:59.065848 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc07d6fd-d23a-4a19-9462-954197f66aeb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:22:59.067735 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc07d6fd-d23a-4a19-9462-954197f66aeb admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:22:59.151892 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1361/2737] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 09:22:58 2022] POST /identity/v3/users => generated 288 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 09:22:59.155723 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8c9618-d18e-4820-a26f-216dcbc1a646 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:22:59.166741 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8c9618-d18e-4820-a26f-216dcbc1a646 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:22:59.171281 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be49691-15db-4bcf-8c94-9890afa0c509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:22:59.171880 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be49691-15db-4bcf-8c94-9890afa0c509 admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:22:59.172163 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be49691-15db-4bcf-8c94-9890afa0c509 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:22:59.172434 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be49691-15db-4bcf-8c94-9890afa0c509 admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:22:59.173099 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be49691-15db-4bcf-8c94-9890afa0c509 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:22:59.174693 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be49691-15db-4bcf-8c94-9890afa0c509 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:22:59.185227 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1377/2738] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 09:22:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 09:22:59.189181 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc07d6fd-d23a-4a19-9462-954197f66aeb admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:22:59.200052 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc07d6fd-d23a-4a19-9462-954197f66aeb admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:22:59.205605 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6201ca7a-2da0-4b35-906f-29a16eed7abd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:22:59.205605 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6201ca7a-2da0-4b35-906f-29a16eed7abd admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:22:59.205605 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6201ca7a-2da0-4b35-906f-29a16eed7abd admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:22:59.205885 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6201ca7a-2da0-4b35-906f-29a16eed7abd admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:22:59.206521 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6201ca7a-2da0-4b35-906f-29a16eed7abd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:22:59.208369 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6201ca7a-2da0-4b35-906f-29a16eed7abd admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:22:59.219357 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1362/2739] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 09:22:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 09:22:59.223272 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be49691-15db-4bcf-8c94-9890afa0c509 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:22:59.233342 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be49691-15db-4bcf-8c94-9890afa0c509 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:22:59.237986 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4fa730-a7dc-4d66-99da-d38a0f474793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:22:59.238633 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4fa730-a7dc-4d66-99da-d38a0f474793 admin admin] REQUEST_METHOD: `PUT` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:22:59.238911 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4fa730-a7dc-4d66-99da-d38a0f474793 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:22:59.239180 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4fa730-a7dc-4d66-99da-d38a0f474793 admin admin] PATH_INFO: `/v3/projects/8aa81851f7054e83a552e645c4b7032a/users/55f9149295b34ca19ca6a052270b7aa5/roles/44b64db484f24a46bf0b7ced46b519bf` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:22:59.254316 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4fa730-a7dc-4d66-99da-d38a0f474793 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8aa81851f7054e83a552e645c4b7032a, user_id=55f9149295b34ca19ca6a052270b7aa5, role_id=44b64db484f24a46bf0b7ced46b519bf)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:22:59.257098 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4fa730-a7dc-4d66-99da-d38a0f474793 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:22:59.327553 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1378/2740] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 09:22:59 2022] PUT /identity/v3/projects/8aa81851f7054e83a552e645c4b7032a/users/55f9149295b34ca19ca6a052270b7aa5/roles/44b64db484f24a46bf0b7ced46b519bf => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:22:59.339388 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c85d975-1612-4b20-ae13-1adee5830593 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:22:59.339699 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c85d975-1612-4b20-ae13-1adee5830593 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:22:59.339972 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c85d975-1612-4b20-ae13-1adee5830593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:22:59.368248 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-3c85d975-1612-4b20-ae13-1adee5830593 None None] MFA Rules not processed for user `55f9149295b34ca19ca6a052270b7aa5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:22:59.369421 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-3c85d975-1612-4b20-ae13-1adee5830593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:22:59.434688 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1363/2741] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 09:22:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:22:59.439104 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4fa730-a7dc-4d66-99da-d38a0f474793 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:22:59.499659 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4fa730-a7dc-4d66-99da-d38a0f474793 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:22:59.504517 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e11ceaa-a15f-4885-8f3a-64afb9da0eb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:22:59.505147 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e11ceaa-a15f-4885-8f3a-64afb9da0eb1 admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:22:59.505456 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e11ceaa-a15f-4885-8f3a-64afb9da0eb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:22:59.505732 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e11ceaa-a15f-4885-8f3a-64afb9da0eb1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:22:59.506357 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e11ceaa-a15f-4885-8f3a-64afb9da0eb1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:22:59.508534 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e11ceaa-a15f-4885-8f3a-64afb9da0eb1 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:22:59.585092 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1379/2742] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 09:22:59 2022] POST /identity/v3/projects => generated 375 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 09:22:59.589520 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c85d975-1612-4b20-ae13-1adee5830593 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:22:59.600191 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c85d975-1612-4b20-ae13-1adee5830593 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:22:59.658197 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed16b540-b035-4706-abbd-bd27c12ab044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:22:59.658696 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed16b540-b035-4706-abbd-bd27c12ab044 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:22:59.658973 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed16b540-b035-4706-abbd-bd27c12ab044 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:22:59.659242 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed16b540-b035-4706-abbd-bd27c12ab044 admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:22:59.659894 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed16b540-b035-4706-abbd-bd27c12ab044 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:22:59.661847 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed16b540-b035-4706-abbd-bd27c12ab044 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:22:59.733400 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1364/2743] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 09:22:59 2022] POST /identity/v3/users => generated 294 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 09:22:59.737479 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e11ceaa-a15f-4885-8f3a-64afb9da0eb1 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:22:59.748637 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e11ceaa-a15f-4885-8f3a-64afb9da0eb1 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:22:59.753207 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5df6da-4ac7-4f8c-929c-ddeeacfe4192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:22:59.753871 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5df6da-4ac7-4f8c-929c-ddeeacfe4192 admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:22:59.754164 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5df6da-4ac7-4f8c-929c-ddeeacfe4192 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:22:59.754430 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5df6da-4ac7-4f8c-929c-ddeeacfe4192 admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:22:59.755060 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5df6da-4ac7-4f8c-929c-ddeeacfe4192 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:22:59.756643 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5df6da-4ac7-4f8c-929c-ddeeacfe4192 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:22:59.767795 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1380/2744] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 09:22:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 09:22:59.771871 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed16b540-b035-4706-abbd-bd27c12ab044 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:22:59.781991 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed16b540-b035-4706-abbd-bd27c12ab044 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:22:59.786853 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17fe42e-ec8d-40e3-b613-68020dd1e698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:22:59.787545 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17fe42e-ec8d-40e3-b613-68020dd1e698 admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:22:59.787824 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17fe42e-ec8d-40e3-b613-68020dd1e698 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:22:59.788118 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17fe42e-ec8d-40e3-b613-68020dd1e698 admin admin] PATH_INFO: `/v3/projects/b2696400ced848a4b1df0ce3bae57a22/users/d986a69b5b794c7fb90ec8827831a656/roles/ec3751692d9c4ecf81b4a71c936c46be` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:22:59.806220 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17fe42e-ec8d-40e3-b613-68020dd1e698 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2696400ced848a4b1df0ce3bae57a22, user_id=d986a69b5b794c7fb90ec8827831a656, role_id=ec3751692d9c4ecf81b4a71c936c46be)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:22:59.808883 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17fe42e-ec8d-40e3-b613-68020dd1e698 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:22:59.811246 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:22:59.889500 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1365/2745] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 09:22:59 2022] PUT /identity/v3/projects/b2696400ced848a4b1df0ce3bae57a22/users/d986a69b5b794c7fb90ec8827831a656/roles/ec3751692d9c4ecf81b4a71c936c46be => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:22:59.893434 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5df6da-4ac7-4f8c-929c-ddeeacfe4192 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:22:59.904541 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5df6da-4ac7-4f8c-929c-ddeeacfe4192 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:22:59.924327 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:22:59.965881 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecb75ca-a2a1-4168-a26e-80a445df2c5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:22:59.966542 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecb75ca-a2a1-4168-a26e-80a445df2c5d admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:22:59.967010 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecb75ca-a2a1-4168-a26e-80a445df2c5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:22:59.967437 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecb75ca-a2a1-4168-a26e-80a445df2c5d admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:22:59.968503 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecb75ca-a2a1-4168-a26e-80a445df2c5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:22:59.970275 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecb75ca-a2a1-4168-a26e-80a445df2c5d admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:22:59.979761 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1381/2746] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 09:22:59 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 09:22:59.984129 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17fe42e-ec8d-40e3-b613-68020dd1e698 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:22:59.994951 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17fe42e-ec8d-40e3-b613-68020dd1e698 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:22:59.999558 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc7ccae-1939-45a0-a3b9-418a63729b85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:23:00.000192 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc7ccae-1939-45a0-a3b9-418a63729b85 admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:23:00.000468 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc7ccae-1939-45a0-a3b9-418a63729b85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:23:00.000741 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc7ccae-1939-45a0-a3b9-418a63729b85 admin admin] PATH_INFO: `/v3/projects/b2696400ced848a4b1df0ce3bae57a22/users/d986a69b5b794c7fb90ec8827831a656/roles/ec3751692d9c4ecf81b4a71c936c46be` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:23:00.002659 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc7ccae-1939-45a0-a3b9-418a63729b85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2696400ced848a4b1df0ce3bae57a22, user_id=d986a69b5b794c7fb90ec8827831a656, role_id=ec3751692d9c4ecf81b4a71c936c46be)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:23:00.005325 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc7ccae-1939-45a0-a3b9-418a63729b85 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:23:00.016516 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1366/2747] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 09:22:59 2022] PUT /identity/v3/projects/b2696400ced848a4b1df0ce3bae57a22/users/d986a69b5b794c7fb90ec8827831a656/roles/ec3751692d9c4ecf81b4a71c936c46be => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:23:00.026533 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bede96b0-59c3-4278-95a5-f9c14016311f None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:23:00.026812 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bede96b0-59c3-4278-95a5-f9c14016311f None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:23:00.027080 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bede96b0-59c3-4278-95a5-f9c14016311f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:23:00.055200 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-bede96b0-59c3-4278-95a5-f9c14016311f None None] MFA Rules not processed for user `d986a69b5b794c7fb90ec8827831a656`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:23:00.056432 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-bede96b0-59c3-4278-95a5-f9c14016311f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:23:00.123009 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1382/2748] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 09:23:00 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:23:00.133890 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc7ccae-1939-45a0-a3b9-418a63729b85 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:23:00.135561 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-ffc7ccae-1939-45a0-a3b9-418a63729b85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:23:00.194032 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc7ccae-1939-45a0-a3b9-418a63729b85 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:23:00.198181 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ade355-a2ef-41be-8932-ca832c56f396 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ba74e28ced4cbba857f54222dfbe41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:23:00.198767 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ade355-a2ef-41be-8932-ca832c56f396 service cinder] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:23:00.199039 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ade355-a2ef-41be-8932-ca832c56f396 service cinder] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:23:00.199300 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ade355-a2ef-41be-8932-ca832c56f396 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:23:00.255330 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ade355-a2ef-41be-8932-ca832c56f396 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:23:00.256776 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ade355-a2ef-41be-8932-ca832c56f396 service cinder] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:23:00.265563 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1367/2749] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:23:00 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:23:04.290738 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bede96b0-59c3-4278-95a5-f9c14016311f None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:23:04.353215 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bede96b0-59c3-4278-95a5-f9c14016311f None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:23:04.358640 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac8ea9c-de57-4625-8713-366bca0fc99c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:23:04.359265 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac8ea9c-de57-4625-8713-366bca0fc99c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:23:04.359541 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac8ea9c-de57-4625-8713-366bca0fc99c admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:23:04.359808 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac8ea9c-de57-4625-8713-366bca0fc99c admin admin] PATH_INFO: `/v3/users/55f9149295b34ca19ca6a052270b7aa5` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:23:04.360979 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac8ea9c-de57-4625-8713-366bca0fc99c admin admin] RBAC: Authorizing `identity:delete_user(user_id=55f9149295b34ca19ca6a052270b7aa5)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:23:04.362899 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac8ea9c-de57-4625-8713-366bca0fc99c admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:23:04.606084 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-dac8ea9c-de57-4625-8713-366bca0fc99c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55f9149295b34ca19ca6a052270b7aa5'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:23:04.607188 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:23:04.618857 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-dac8ea9c-de57-4625-8713-366bca0fc99c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55f9149295b34ca19ca6a052270b7aa5'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:23:04.625704 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-dac8ea9c-de57-4625-8713-366bca0fc99c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55f9149295b34ca19ca6a052270b7aa5'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:23:04.716130 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-dac8ea9c-de57-4625-8713-366bca0fc99c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55f9149295b34ca19ca6a052270b7aa5'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:23:04.716739 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-dac8ea9c-de57-4625-8713-366bca0fc99c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55f9149295b34ca19ca6a052270b7aa5'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:23:04.722788 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1383/2750] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 09:23:04 2022] DELETE /identity/v3/users/55f9149295b34ca19ca6a052270b7aa5 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:23:04.727785 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa37241-5555-4bde-8732-00cacaf9633f None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:23:04.728274 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa37241-5555-4bde-8732-00cacaf9633f None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:23:04.728374 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa37241-5555-4bde-8732-00cacaf9633f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:23:04.747219 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-bfa37241-5555-4bde-8732-00cacaf9633f None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:23:04.748479 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-bfa37241-5555-4bde-8732-00cacaf9633f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:23:04.806964 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1368/2751] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 09:23:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:23:04.816750 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac8ea9c-de57-4625-8713-366bca0fc99c admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:23:04.818959 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-dac8ea9c-de57-4625-8713-366bca0fc99c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:23:04.874447 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac8ea9c-de57-4625-8713-366bca0fc99c admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:23:04.878033 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c87a7bc-a48b-42c6-ae5f-f41c741f665b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:23:04.878628 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c87a7bc-a48b-42c6-ae5f-f41c741f665b service neutron] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:23:04.878908 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c87a7bc-a48b-42c6-ae5f-f41c741f665b service neutron] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:23:04.879176 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c87a7bc-a48b-42c6-ae5f-f41c741f665b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:23:04.889061 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c87a7bc-a48b-42c6-ae5f-f41c741f665b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:23:04.890486 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c87a7bc-a48b-42c6-ae5f-f41c741f665b service neutron] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:23:04.900280 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1384/2752] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:23:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:23:05.312641 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa37241-5555-4bde-8732-00cacaf9633f None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:23:05.314114 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-bfa37241-5555-4bde-8732-00cacaf9633f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:23:05.326809 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa37241-5555-4bde-8732-00cacaf9633f None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:23:05.331907 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1946ce-d231-4fdb-abfe-cb90dadb1863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:23:05.332537 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1946ce-d231-4fdb-abfe-cb90dadb1863 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:23:05.332879 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1946ce-d231-4fdb-abfe-cb90dadb1863 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:23:05.333286 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1946ce-d231-4fdb-abfe-cb90dadb1863 admin admin] PATH_INFO: `/v3/projects/8aa81851f7054e83a552e645c4b7032a` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:23:05.334358 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1946ce-d231-4fdb-abfe-cb90dadb1863 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8aa81851f7054e83a552e645c4b7032a)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:23:05.336298 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1946ce-d231-4fdb-abfe-cb90dadb1863 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:23:05.430853 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:INSERT=2,keystone:SELECT=96,keystone:DELETE=2 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:23:05.444019 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-3f1946ce-d231-4fdb-abfe-cb90dadb1863 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8aa81851f7054e83a552e645c4b7032a', 'request_id': 'req-3f1946ce-d231-4fdb-abfe-cb90dadb1863'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:23:05.550244 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-3f1946ce-d231-4fdb-abfe-cb90dadb1863 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8aa81851f7054e83a552e645c4b7032a', 'request_id': 'req-3f1946ce-d231-4fdb-abfe-cb90dadb1863'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:23:05.556901 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-3f1946ce-d231-4fdb-abfe-cb90dadb1863 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8aa81851f7054e83a552e645c4b7032a', 'request_id': 'req-3f1946ce-d231-4fdb-abfe-cb90dadb1863'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:23:05.568221 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-3f1946ce-d231-4fdb-abfe-cb90dadb1863 admin admin] The token cache is being invalidate because project 8aa81851f7054e83a552e645c4b7032a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 09:23:05.568511 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-3f1946ce-d231-4fdb-abfe-cb90dadb1863 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:23:05.569296 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-3f1946ce-d231-4fdb-abfe-cb90dadb1863 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:23:05.571255 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1369/2753] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 09:23:05 2022] DELETE /identity/v3/projects/8aa81851f7054e83a552e645c4b7032a => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:23:05.575833 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c87a7bc-a48b-42c6-ae5f-f41c741f665b service neutron] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:23:05.586973 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c87a7bc-a48b-42c6-ae5f-f41c741f665b service neutron] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:23:05.643324 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dc888d-9277-483d-8917-c8de9cc59223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:23:05.643987 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dc888d-9277-483d-8917-c8de9cc59223 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:23:05.644270 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dc888d-9277-483d-8917-c8de9cc59223 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:23:05.644544 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dc888d-9277-483d-8917-c8de9cc59223 admin admin] PATH_INFO: `/v3/users/d986a69b5b794c7fb90ec8827831a656` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:23:05.645786 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dc888d-9277-483d-8917-c8de9cc59223 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d986a69b5b794c7fb90ec8827831a656)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:23:05.647831 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dc888d-9277-483d-8917-c8de9cc59223 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:23:05.832977 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-33dc888d-9277-483d-8917-c8de9cc59223 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd986a69b5b794c7fb90ec8827831a656'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:23:05.844960 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-33dc888d-9277-483d-8917-c8de9cc59223 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd986a69b5b794c7fb90ec8827831a656'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:23:05.849353 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:23:05.852462 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-33dc888d-9277-483d-8917-c8de9cc59223 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd986a69b5b794c7fb90ec8827831a656'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:23:05.950961 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-33dc888d-9277-483d-8917-c8de9cc59223 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd986a69b5b794c7fb90ec8827831a656'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:23:05.951612 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-33dc888d-9277-483d-8917-c8de9cc59223 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd986a69b5b794c7fb90ec8827831a656'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:23:05.957855 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1385/2754] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 09:23:05 2022] DELETE /identity/v3/users/d986a69b5b794c7fb90ec8827831a656 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:23:06.381183 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1946ce-d231-4fdb-abfe-cb90dadb1863 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:23:06.387279 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-3f1946ce-d231-4fdb-abfe-cb90dadb1863 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:23:06.398744 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1946ce-d231-4fdb-abfe-cb90dadb1863 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:23:06.467176 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7e652e-3ff1-41d2-9ce2-98235329f940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:23:06.468162 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7e652e-3ff1-41d2-9ce2-98235329f940 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:23:06.468528 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7e652e-3ff1-41d2-9ce2-98235329f940 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:23:06.468804 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7e652e-3ff1-41d2-9ce2-98235329f940 admin admin] PATH_INFO: `/v3/projects/b2696400ced848a4b1df0ce3bae57a22` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:23:06.470005 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7e652e-3ff1-41d2-9ce2-98235329f940 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2696400ced848a4b1df0ce3bae57a22)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:23:06.471960 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7e652e-3ff1-41d2-9ce2-98235329f940 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:23:06.581452 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-ce7e652e-3ff1-41d2-9ce2-98235329f940 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2696400ced848a4b1df0ce3bae57a22', 'request_id': 'req-ce7e652e-3ff1-41d2-9ce2-98235329f940'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:23:06.642457 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-ce7e652e-3ff1-41d2-9ce2-98235329f940 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2696400ced848a4b1df0ce3bae57a22', 'request_id': 'req-ce7e652e-3ff1-41d2-9ce2-98235329f940'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:23:06.649476 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-ce7e652e-3ff1-41d2-9ce2-98235329f940 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2696400ced848a4b1df0ce3bae57a22', 'request_id': 'req-ce7e652e-3ff1-41d2-9ce2-98235329f940'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:23:06.660986 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-ce7e652e-3ff1-41d2-9ce2-98235329f940 admin admin] The token cache is being invalidate because project b2696400ced848a4b1df0ce3bae57a22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 09:23:06.661273 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-ce7e652e-3ff1-41d2-9ce2-98235329f940 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:23:06.661918 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-ce7e652e-3ff1-41d2-9ce2-98235329f940 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:23:06.664307 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1370/2755] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 09:23:06 2022] DELETE /identity/v3/projects/b2696400ced848a4b1df0ce3bae57a22 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:23:06.670559 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e415e1-5384-48b4-ad32-b0017e4011c4 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:23:06.671088 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e415e1-5384-48b4-ad32-b0017e4011c4 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:23:06.671184 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e415e1-5384-48b4-ad32-b0017e4011c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:23:06.692531 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-b4e415e1-5384-48b4-ad32-b0017e4011c4 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:23:06.693748 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-b4e415e1-5384-48b4-ad32-b0017e4011c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:23:06.756997 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1386/2756] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 09:23:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:23:06.766731 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbf560b-888a-4eeb-b8bb-472ccbdcf1c1 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:23:06.767009 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbf560b-888a-4eeb-b8bb-472ccbdcf1c1 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:23:06.767276 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbf560b-888a-4eeb-b8bb-472ccbdcf1c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:23:06.789445 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-5dbf560b-888a-4eeb-b8bb-472ccbdcf1c1 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:23:06.790654 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-5dbf560b-888a-4eeb-b8bb-472ccbdcf1c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:23:06.799870 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1371/2757] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 09:23:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:23:06.805522 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e415e1-5384-48b4-ad32-b0017e4011c4 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:23:06.815736 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e415e1-5384-48b4-ad32-b0017e4011c4 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:23:06.821046 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5f0606-1f3e-4c56-947a-c48dffe7192e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:23:06.821648 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5f0606-1f3e-4c56-947a-c48dffe7192e admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:23:06.821923 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5f0606-1f3e-4c56-947a-c48dffe7192e admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:23:06.822216 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5f0606-1f3e-4c56-947a-c48dffe7192e admin admin] PATH_INFO: `/v3/domains` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:23:06.822854 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5f0606-1f3e-4c56-947a-c48dffe7192e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:23:06.823152 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5f0606-1f3e-4c56-947a-c48dffe7192e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:23:06.824965 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5f0606-1f3e-4c56-947a-c48dffe7192e admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:23:06.840732 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1387/2758] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Aug 10 09:23:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:23:06.845193 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbf560b-888a-4eeb-b8bb-472ccbdcf1c1 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:23:06.854939 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbf560b-888a-4eeb-b8bb-472ccbdcf1c1 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:23:06.859533 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370c2655-8ba0-4eef-b5fe-1dacaf2f7ca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:23:06.860119 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370c2655-8ba0-4eef-b5fe-1dacaf2f7ca4 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:23:06.860397 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370c2655-8ba0-4eef-b5fe-1dacaf2f7ca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:23:06.860663 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370c2655-8ba0-4eef-b5fe-1dacaf2f7ca4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:23:06.861337 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370c2655-8ba0-4eef-b5fe-1dacaf2f7ca4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:23:06.863216 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370c2655-8ba0-4eef-b5fe-1dacaf2f7ca4 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:23:06.928850 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1372/2759] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 09:23:06 2022] POST /identity/v3/projects => generated 373 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 09:23:06.933552 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5f0606-1f3e-4c56-947a-c48dffe7192e admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:23:06.943529 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5f0606-1f3e-4c56-947a-c48dffe7192e admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:23:06.998446 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944703ed-5b7e-4454-85b1-4365ed2f0c05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:23:06.999045 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944703ed-5b7e-4454-85b1-4365ed2f0c05 admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:23:06.999321 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944703ed-5b7e-4454-85b1-4365ed2f0c05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:23:06.999585 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944703ed-5b7e-4454-85b1-4365ed2f0c05 admin admin] PATH_INFO: `/v3/users` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:23:07.000193 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944703ed-5b7e-4454-85b1-4365ed2f0c05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:23:07.002122 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944703ed-5b7e-4454-85b1-4365ed2f0c05 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:23:07.093224 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1388/2760] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 09:23:06 2022] POST /identity/v3/users => generated 287 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 09:23:07.097230 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370c2655-8ba0-4eef-b5fe-1dacaf2f7ca4 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:23:07.108385 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370c2655-8ba0-4eef-b5fe-1dacaf2f7ca4 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:23:07.113181 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc835d07-49aa-4338-a35d-3f3a0f7cc47a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:23:07.113782 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc835d07-49aa-4338-a35d-3f3a0f7cc47a admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:23:07.114089 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc835d07-49aa-4338-a35d-3f3a0f7cc47a admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:23:07.114362 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc835d07-49aa-4338-a35d-3f3a0f7cc47a admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:23:07.115007 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc835d07-49aa-4338-a35d-3f3a0f7cc47a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:23:07.116611 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc835d07-49aa-4338-a35d-3f3a0f7cc47a admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:23:07.127926 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1373/2761] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 09:23:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 09:23:07.131991 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944703ed-5b7e-4454-85b1-4365ed2f0c05 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:23:07.142406 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944703ed-5b7e-4454-85b1-4365ed2f0c05 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:23:07.147053 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd6e338-1b1f-41c5-990e-0ce69cba0ab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:23:07.147662 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd6e338-1b1f-41c5-990e-0ce69cba0ab9 admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:23:07.147943 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd6e338-1b1f-41c5-990e-0ce69cba0ab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:23:07.148222 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd6e338-1b1f-41c5-990e-0ce69cba0ab9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:23:07.148863 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd6e338-1b1f-41c5-990e-0ce69cba0ab9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:23:07.150653 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd6e338-1b1f-41c5-990e-0ce69cba0ab9 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:23:07.161313 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1389/2762] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 09:23:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 09:23:07.166517 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc835d07-49aa-4338-a35d-3f3a0f7cc47a admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:23:07.176846 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc835d07-49aa-4338-a35d-3f3a0f7cc47a admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:23:07.190799 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed3be9c-1e53-4792-9e4e-4e32030c366a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:23:07.190799 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed3be9c-1e53-4792-9e4e-4e32030c366a admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:23:07.190799 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed3be9c-1e53-4792-9e4e-4e32030c366a admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:23:07.190799 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed3be9c-1e53-4792-9e4e-4e32030c366a admin admin] PATH_INFO: `/v3/projects/e1616e4f3f89412f9a3c631208ecb493/users/d113e0f5215c44699b8eb744ad712735/roles/44b64db484f24a46bf0b7ced46b519bf` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:23:07.191960 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:23:07.206615 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed3be9c-1e53-4792-9e4e-4e32030c366a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1616e4f3f89412f9a3c631208ecb493, user_id=d113e0f5215c44699b8eb744ad712735, role_id=44b64db484f24a46bf0b7ced46b519bf)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:23:07.209417 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed3be9c-1e53-4792-9e4e-4e32030c366a admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:23:07.305858 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1374/2763] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 09:23:07 2022] PUT /identity/v3/projects/e1616e4f3f89412f9a3c631208ecb493/users/d113e0f5215c44699b8eb744ad712735/roles/44b64db484f24a46bf0b7ced46b519bf => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 09:23:07.310874 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b9f4f2-a83c-4ca0-9291-bc1c62069b62 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:23:07.311179 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b9f4f2-a83c-4ca0-9291-bc1c62069b62 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:23:07.311482 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b9f4f2-a83c-4ca0-9291-bc1c62069b62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:23:07.331910 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-63b9f4f2-a83c-4ca0-9291-bc1c62069b62 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:23:07.333135 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-63b9f4f2-a83c-4ca0-9291-bc1c62069b62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:23:07.378784 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:23:07.391855 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1390/2764] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 09:23:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:23:07.402376 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed3be9c-1e53-4792-9e4e-4e32030c366a admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:23:07.404127 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-bed3be9c-1e53-4792-9e4e-4e32030c366a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:23:07.461982 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed3be9c-1e53-4792-9e4e-4e32030c366a admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:23:07.465657 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc59df2a-32f1-4960-9370-053ceb9024d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:23:07.466255 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc59df2a-32f1-4960-9370-053ceb9024d1 service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:23:07.466533 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc59df2a-32f1-4960-9370-053ceb9024d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:23:07.466809 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc59df2a-32f1-4960-9370-053ceb9024d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:23:07.476337 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc59df2a-32f1-4960-9370-053ceb9024d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:23:07.477996 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc59df2a-32f1-4960-9370-053ceb9024d1 service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:23:07.487794 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1375/2765] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:23:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:23:08.733868 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37015644-5a2e-4219-a971-590d2f17db1f None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:23:08.735044 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37015644-5a2e-4219-a971-590d2f17db1f None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:23:08.735348 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37015644-5a2e-4219-a971-590d2f17db1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:23:08.765787 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-37015644-5a2e-4219-a971-590d2f17db1f None None] MFA Rules not processed for user `d113e0f5215c44699b8eb744ad712735`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:23:08.766993 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-37015644-5a2e-4219-a971-590d2f17db1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:23:08.833201 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1391/2766] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 09:23:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:23:08.844541 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc59df2a-32f1-4960-9370-053ceb9024d1 service neutron] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:23:08.846017 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-bc59df2a-32f1-4960-9370-053ceb9024d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:23:08.907442 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc59df2a-32f1-4960-9370-053ceb9024d1 service neutron] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:23:08.911572 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9110558-026b-4b81-888c-98001888b824 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ba74e28ced4cbba857f54222dfbe41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:23:08.912167 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9110558-026b-4b81-888c-98001888b824 service cinder] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:23:08.913651 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9110558-026b-4b81-888c-98001888b824 service cinder] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:23:08.913651 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9110558-026b-4b81-888c-98001888b824 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:23:08.923406 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9110558-026b-4b81-888c-98001888b824 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:23:08.924873 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9110558-026b-4b81-888c-98001888b824 service cinder] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:23:08.935337 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1376/2767] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:23:08 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:23:14.003153 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0655c2-468c-421b-9de3-545c61ad4147 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:23:14.003708 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0655c2-468c-421b-9de3-545c61ad4147 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:23:14.003708 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0655c2-468c-421b-9de3-545c61ad4147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:23:14.022295 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-3f0655c2-468c-421b-9de3-545c61ad4147 None None] MFA Rules not processed for user `d113e0f5215c44699b8eb744ad712735`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:23:14.024280 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-3f0655c2-468c-421b-9de3-545c61ad4147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:23:14.031153 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1392/2768] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 09:23:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:23:14.079700 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9110558-026b-4b81-888c-98001888b824 service cinder] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:23:14.081222 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-f9110558-026b-4b81-888c-98001888b824 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:23:14.137491 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9110558-026b-4b81-888c-98001888b824 service cinder] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:23:14.141056 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95297708-fcdc-43bc-bff3-f324d09156cb service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3263af99fc2f48e79fd1e5b8a1e555a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:23:14.141653 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95297708-fcdc-43bc-bff3-f324d09156cb service glance] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:23:14.141933 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95297708-fcdc-43bc-bff3-f324d09156cb service glance] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:23:14.142208 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95297708-fcdc-43bc-bff3-f324d09156cb service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:23:14.143911 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-95297708-fcdc-43bc-bff3-f324d09156cb service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:23:14.188631 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:23:14.231694 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95297708-fcdc-43bc-bff3-f324d09156cb service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:23:14.233146 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95297708-fcdc-43bc-bff3-f324d09156cb service glance] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:23:14.252827 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1377/2769] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:23:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:23:14.456885 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0655c2-468c-421b-9de3-545c61ad4147 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:23:14.458332 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-3f0655c2-468c-421b-9de3-545c61ad4147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:23:14.471073 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0655c2-468c-421b-9de3-545c61ad4147 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:23:14.474658 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047a3e10-4076-4c45-bcb7-6b65855416a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:23:14.475302 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047a3e10-4076-4c45-bcb7-6b65855416a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:23:14.483195 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047a3e10-4076-4c45-bcb7-6b65855416a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:23:14.483195 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047a3e10-4076-4c45-bcb7-6b65855416a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:23:14.483195 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-047a3e10-4076-4c45-bcb7-6b65855416a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:23:14.487697 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047a3e10-4076-4c45-bcb7-6b65855416a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:23:14.489155 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047a3e10-4076-4c45-bcb7-6b65855416a1 service neutron] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:23:14.497507 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1393/2770] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:23:14 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:23:15.914695 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95297708-fcdc-43bc-bff3-f324d09156cb service glance] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:23:15.917810 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-95297708-fcdc-43bc-bff3-f324d09156cb service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:23:15.924866 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047a3e10-4076-4c45-bcb7-6b65855416a1 service neutron] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:23:15.934124 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047a3e10-4076-4c45-bcb7-6b65855416a1 service neutron] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:23:15.937797 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30856660-a487-41c2-aab1-41e69690bdc2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:23:15.938419 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30856660-a487-41c2-aab1-41e69690bdc2 service neutron] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:23:15.938696 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30856660-a487-41c2-aab1-41e69690bdc2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:23:15.938969 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30856660-a487-41c2-aab1-41e69690bdc2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:23:15.941244 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-30856660-a487-41c2-aab1-41e69690bdc2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:23:15.956065 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30856660-a487-41c2-aab1-41e69690bdc2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:23:15.957606 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30856660-a487-41c2-aab1-41e69690bdc2 service neutron] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:23:15.969034 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1394/2771] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:23:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 09:23:15.988717 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95297708-fcdc-43bc-bff3-f324d09156cb service glance] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:23:15.994608 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b1f89f-92ef-4b59-bb5a-970e15676105 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2912307c56594125b157f1cfdf7c96a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:23:15.995216 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b1f89f-92ef-4b59-bb5a-970e15676105 service placement] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:23:15.995496 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b1f89f-92ef-4b59-bb5a-970e15676105 service placement] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:23:15.995772 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b1f89f-92ef-4b59-bb5a-970e15676105 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:23:15.997595 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-85b1f89f-92ef-4b59-bb5a-970e15676105 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:23:16.010005 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b1f89f-92ef-4b59-bb5a-970e15676105 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:23:16.011588 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b1f89f-92ef-4b59-bb5a-970e15676105 service placement] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:23:16.021883 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1378/2772] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:23:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:23:19.163259 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30856660-a487-41c2-aab1-41e69690bdc2 service neutron] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:23:19.172985 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30856660-a487-41c2-aab1-41e69690bdc2 service neutron] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:23:19.176549 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377155ac-5cf4-4d60-b65e-8b772f94f5ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:23:19.177166 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377155ac-5cf4-4d60-b65e-8b772f94f5ce service neutron] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:23:19.177448 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377155ac-5cf4-4d60-b65e-8b772f94f5ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:23:19.177724 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377155ac-5cf4-4d60-b65e-8b772f94f5ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:23:19.179416 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-377155ac-5cf4-4d60-b65e-8b772f94f5ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:23:19.189682 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377155ac-5cf4-4d60-b65e-8b772f94f5ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:23:19.191118 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377155ac-5cf4-4d60-b65e-8b772f94f5ce service neutron] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:23:19.199204 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1395/2773] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:23:19 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:23:20.003316 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b1f89f-92ef-4b59-bb5a-970e15676105 service placement] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:23:20.004798 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-85b1f89f-92ef-4b59-bb5a-970e15676105 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:23:20.020271 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b1f89f-92ef-4b59-bb5a-970e15676105 service placement] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:23:20.026407 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d6bafb-e909-4bc2-8a98-aef8e7109d34 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a943008193914f76a663d489bd73e873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:23:20.027008 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d6bafb-e909-4bc2-8a98-aef8e7109d34 service nova] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:23:20.027286 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d6bafb-e909-4bc2-8a98-aef8e7109d34 service nova] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:23:20.027557 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d6bafb-e909-4bc2-8a98-aef8e7109d34 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:23:20.029307 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-09d6bafb-e909-4bc2-8a98-aef8e7109d34 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:23:20.042842 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d6bafb-e909-4bc2-8a98-aef8e7109d34 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:23:20.044350 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d6bafb-e909-4bc2-8a98-aef8e7109d34 service nova] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:23:20.055579 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1379/2774] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:23:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:23:29.195925 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=56 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:23:30.050435 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=63 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:23:30.908387 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377155ac-5cf4-4d60-b65e-8b772f94f5ce service neutron] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:23:30.909971 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-377155ac-5cf4-4d60-b65e-8b772f94f5ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:23:30.926675 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377155ac-5cf4-4d60-b65e-8b772f94f5ce service neutron] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:23:30.932928 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b32c9cb-96e3-4302-b72f-036a0723fcb7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a943008193914f76a663d489bd73e873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:23:30.933615 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b32c9cb-96e3-4302-b72f-036a0723fcb7 service nova] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:23:30.933898 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b32c9cb-96e3-4302-b72f-036a0723fcb7 service nova] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:23:30.934171 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b32c9cb-96e3-4302-b72f-036a0723fcb7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:23:30.935870 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-7b32c9cb-96e3-4302-b72f-036a0723fcb7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:23:30.948029 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b32c9cb-96e3-4302-b72f-036a0723fcb7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:23:30.949660 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b32c9cb-96e3-4302-b72f-036a0723fcb7 service nova] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:23:30.960366 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1396/2775] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:23:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:23:31.345598 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d6bafb-e909-4bc2-8a98-aef8e7109d34 service nova] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:23:31.347263 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-09d6bafb-e909-4bc2-8a98-aef8e7109d34 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:23:31.361895 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d6bafb-e909-4bc2-8a98-aef8e7109d34 service nova] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:23:31.366797 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab8362c-6e86-4078-920a-f8356a16d4b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:23:31.368018 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab8362c-6e86-4078-920a-f8356a16d4b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:23:31.368859 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab8362c-6e86-4078-920a-f8356a16d4b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:23:31.369914 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab8362c-6e86-4078-920a-f8356a16d4b5 admin admin] PATH_INFO: `/v3/users/8409645cd3c14e83a305654ba9b18f88` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:23:31.372019 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab8362c-6e86-4078-920a-f8356a16d4b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8409645cd3c14e83a305654ba9b18f88)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:23:31.374400 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab8362c-6e86-4078-920a-f8356a16d4b5 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:23:31.508313 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-7ab8362c-6e86-4078-920a-f8356a16d4b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8409645cd3c14e83a305654ba9b18f88'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:23:31.522805 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-7ab8362c-6e86-4078-920a-f8356a16d4b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8409645cd3c14e83a305654ba9b18f88'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:23:31.530317 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-7ab8362c-6e86-4078-920a-f8356a16d4b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8409645cd3c14e83a305654ba9b18f88'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:23:31.531030 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-7ab8362c-6e86-4078-920a-f8356a16d4b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8409645cd3c14e83a305654ba9b18f88'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:23:31.531665 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-7ab8362c-6e86-4078-920a-f8356a16d4b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8409645cd3c14e83a305654ba9b18f88'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:23:31.584868 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1380/2776] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 09:23:31 2022] DELETE /identity/v3/users/8409645cd3c14e83a305654ba9b18f88 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:23:31.590365 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5650f1-8764-4a4b-bcbc-52b785abc3c0 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:23:31.590649 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5650f1-8764-4a4b-bcbc-52b785abc3c0 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:23:31.590916 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5650f1-8764-4a4b-bcbc-52b785abc3c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:23:31.609718 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-4a5650f1-8764-4a4b-bcbc-52b785abc3c0 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:23:31.610889 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-4a5650f1-8764-4a4b-bcbc-52b785abc3c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:23:31.667264 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1397/2777] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 09:23:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:23:31.677321 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab8362c-6e86-4078-920a-f8356a16d4b5 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:23:31.678731 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-7ab8362c-6e86-4078-920a-f8356a16d4b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:23:31.736257 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab8362c-6e86-4078-920a-f8356a16d4b5 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:23:31.739850 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc73ce0-ff84-4de1-b830-04976904c600 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:23:31.740438 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc73ce0-ff84-4de1-b830-04976904c600 service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:23:31.740711 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc73ce0-ff84-4de1-b830-04976904c600 service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:23:31.741008 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc73ce0-ff84-4de1-b830-04976904c600 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:23:31.750150 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc73ce0-ff84-4de1-b830-04976904c600 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:23:31.751549 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc73ce0-ff84-4de1-b830-04976904c600 service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:23:31.761959 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1381/2778] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:23:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:23:32.148510 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5650f1-8764-4a4b-bcbc-52b785abc3c0 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:23:32.150275 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-4a5650f1-8764-4a4b-bcbc-52b785abc3c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:23:32.162614 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5650f1-8764-4a4b-bcbc-52b785abc3c0 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:23:32.175023 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb78e80-c123-4c8d-ad0a-d3e3b0b1f462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:23:32.175023 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb78e80-c123-4c8d-ad0a-d3e3b0b1f462 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:23:32.175023 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb78e80-c123-4c8d-ad0a-d3e3b0b1f462 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:23:32.175023 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb78e80-c123-4c8d-ad0a-d3e3b0b1f462 admin admin] PATH_INFO: `/v3/projects/e9f8c00c05c745358fb8f9e9cf6bc9da` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:23:32.175752 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb78e80-c123-4c8d-ad0a-d3e3b0b1f462 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9f8c00c05c745358fb8f9e9cf6bc9da)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:23:32.175752 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb78e80-c123-4c8d-ad0a-d3e3b0b1f462 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:23:32.282638 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-7fb78e80-c123-4c8d-ad0a-d3e3b0b1f462 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9f8c00c05c745358fb8f9e9cf6bc9da', 'request_id': 'req-7fb78e80-c123-4c8d-ad0a-d3e3b0b1f462'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:23:32.288050 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-7fb78e80-c123-4c8d-ad0a-d3e3b0b1f462 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9f8c00c05c745358fb8f9e9cf6bc9da', 'request_id': 'req-7fb78e80-c123-4c8d-ad0a-d3e3b0b1f462'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:23:32.339856 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-7fb78e80-c123-4c8d-ad0a-d3e3b0b1f462 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9f8c00c05c745358fb8f9e9cf6bc9da', 'request_id': 'req-7fb78e80-c123-4c8d-ad0a-d3e3b0b1f462'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:23:32.351115 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-7fb78e80-c123-4c8d-ad0a-d3e3b0b1f462 admin admin] The token cache is being invalidate because project e9f8c00c05c745358fb8f9e9cf6bc9da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 09:23:32.351394 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-7fb78e80-c123-4c8d-ad0a-d3e3b0b1f462 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:23:32.352966 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-7fb78e80-c123-4c8d-ad0a-d3e3b0b1f462 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:23:32.354452 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1398/2779] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 09:23:32 2022] DELETE /identity/v3/projects/e9f8c00c05c745358fb8f9e9cf6bc9da => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:23:32.359566 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fcf6a4-e6ca-4428-8165-64c1d23a0bda None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:23:32.360044 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fcf6a4-e6ca-4428-8165-64c1d23a0bda None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:23:32.360143 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fcf6a4-e6ca-4428-8165-64c1d23a0bda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:23:32.379823 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-b9fcf6a4-e6ca-4428-8165-64c1d23a0bda None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:23:32.381028 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-b9fcf6a4-e6ca-4428-8165-64c1d23a0bda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:23:32.440840 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1382/2780] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 09:23:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:23:32.448779 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e13d1a-dcd0-4456-b063-2ac27422e247 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:23:32.449096 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e13d1a-dcd0-4456-b063-2ac27422e247 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:23:32.449375 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e13d1a-dcd0-4456-b063-2ac27422e247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:23:32.468633 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-66e13d1a-dcd0-4456-b063-2ac27422e247 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:23:32.469867 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-66e13d1a-dcd0-4456-b063-2ac27422e247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:23:32.478249 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1399/2781] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 09:23:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:23:32.482480 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fcf6a4-e6ca-4428-8165-64c1d23a0bda None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:23:32.491918 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fcf6a4-e6ca-4428-8165-64c1d23a0bda None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:23:32.497639 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda65654-46b7-4852-891d-2bcd8a7af2e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:23:32.498270 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda65654-46b7-4852-891d-2bcd8a7af2e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:23:32.498548 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda65654-46b7-4852-891d-2bcd8a7af2e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:23:32.498848 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda65654-46b7-4852-891d-2bcd8a7af2e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:23:32.499491 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda65654-46b7-4852-891d-2bcd8a7af2e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:23:32.499794 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda65654-46b7-4852-891d-2bcd8a7af2e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:23:32.501428 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda65654-46b7-4852-891d-2bcd8a7af2e5 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:23:32.512762 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1383/2782] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Aug 10 09:23:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:23:32.516836 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e13d1a-dcd0-4456-b063-2ac27422e247 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:23:32.526188 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e13d1a-dcd0-4456-b063-2ac27422e247 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:23:32.530821 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada6e2bb-9df4-4faf-b10c-9b0d64e8782e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:23:32.531413 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada6e2bb-9df4-4faf-b10c-9b0d64e8782e admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:23:32.531692 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada6e2bb-9df4-4faf-b10c-9b0d64e8782e admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:23:32.531965 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada6e2bb-9df4-4faf-b10c-9b0d64e8782e admin admin] PATH_INFO: `/v3/projects` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:23:32.532590 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada6e2bb-9df4-4faf-b10c-9b0d64e8782e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:23:32.534612 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada6e2bb-9df4-4faf-b10c-9b0d64e8782e admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:23:32.625012 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1400/2783] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 09:23:32 2022] POST /identity/v3/projects => generated 385 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 09:23:32.626258 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda65654-46b7-4852-891d-2bcd8a7af2e5 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:23:32.631367 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:23:32.636780 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda65654-46b7-4852-891d-2bcd8a7af2e5 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:23:32.693526 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f90e7ec-4752-4711-9793-e1951923adf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:23:32.697644 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f90e7ec-4752-4711-9793-e1951923adf9 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:23:32.697993 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f90e7ec-4752-4711-9793-e1951923adf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:23:32.698273 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f90e7ec-4752-4711-9793-e1951923adf9 admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:23:32.698905 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f90e7ec-4752-4711-9793-e1951923adf9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:23:32.700819 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f90e7ec-4752-4711-9793-e1951923adf9 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:23:32.785681 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1384/2784] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 09:23:32 2022] POST /identity/v3/users => generated 293 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 09:23:32.790701 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada6e2bb-9df4-4faf-b10c-9b0d64e8782e admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:23:32.801482 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada6e2bb-9df4-4faf-b10c-9b0d64e8782e admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:23:32.807074 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11967d0-5bc2-4e95-9d6a-c35e032fb36f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:23:32.807683 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11967d0-5bc2-4e95-9d6a-c35e032fb36f admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:23:32.807961 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11967d0-5bc2-4e95-9d6a-c35e032fb36f admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:23:32.808228 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11967d0-5bc2-4e95-9d6a-c35e032fb36f admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:23:32.808865 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11967d0-5bc2-4e95-9d6a-c35e032fb36f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:23:32.810475 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11967d0-5bc2-4e95-9d6a-c35e032fb36f admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:23:32.820832 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1401/2785] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 09:23:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 09:23:32.825034 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f90e7ec-4752-4711-9793-e1951923adf9 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:23:32.835128 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f90e7ec-4752-4711-9793-e1951923adf9 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:23:32.840412 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aca5e5a-0a89-4b1c-81b6-912a28447345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:23:32.841037 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aca5e5a-0a89-4b1c-81b6-912a28447345 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:23:32.841318 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aca5e5a-0a89-4b1c-81b6-912a28447345 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:23:32.841588 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aca5e5a-0a89-4b1c-81b6-912a28447345 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:23:32.842226 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aca5e5a-0a89-4b1c-81b6-912a28447345 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:23:32.843806 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aca5e5a-0a89-4b1c-81b6-912a28447345 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:23:32.855070 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1385/2786] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 09:23:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 09:23:32.859302 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11967d0-5bc2-4e95-9d6a-c35e032fb36f admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:23:32.869488 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11967d0-5bc2-4e95-9d6a-c35e032fb36f admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:23:32.874079 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9752c9d9-acc2-43bc-a427-e29220a5461b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:23:32.874719 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9752c9d9-acc2-43bc-a427-e29220a5461b admin admin] REQUEST_METHOD: `PUT` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:23:32.882290 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9752c9d9-acc2-43bc-a427-e29220a5461b admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:23:32.882290 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9752c9d9-acc2-43bc-a427-e29220a5461b admin admin] PATH_INFO: `/v3/projects/15117b97af5f40bbb1ba72a989ba5cab/users/948d87bd5ba04663ac3fa8be3926d67a/roles/44b64db484f24a46bf0b7ced46b519bf` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:23:32.889637 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9752c9d9-acc2-43bc-a427-e29220a5461b admin admin] RBAC: Authorizing `identity:create_grant(project_id=15117b97af5f40bbb1ba72a989ba5cab, user_id=948d87bd5ba04663ac3fa8be3926d67a, role_id=44b64db484f24a46bf0b7ced46b519bf)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:23:32.892292 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9752c9d9-acc2-43bc-a427-e29220a5461b admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:23:32.951543 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1402/2787] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 09:23:32 2022] PUT /identity/v3/projects/15117b97af5f40bbb1ba72a989ba5cab/users/948d87bd5ba04663ac3fa8be3926d67a/roles/44b64db484f24a46bf0b7ced46b519bf => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:23:32.956477 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977d4d04-5c7a-4eab-b42f-c56d7df0c26c None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:23:32.956760 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977d4d04-5c7a-4eab-b42f-c56d7df0c26c None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:23:32.957055 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977d4d04-5c7a-4eab-b42f-c56d7df0c26c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:23:32.976193 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-977d4d04-5c7a-4eab-b42f-c56d7df0c26c None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:23:32.978104 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-977d4d04-5c7a-4eab-b42f-c56d7df0c26c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:23:33.035071 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1386/2788] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 09:23:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:23:33.045608 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9752c9d9-acc2-43bc-a427-e29220a5461b admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:23:33.047341 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-9752c9d9-acc2-43bc-a427-e29220a5461b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:23:33.057003 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:23:33.104486 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9752c9d9-acc2-43bc-a427-e29220a5461b admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:23:33.108893 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d824045-4df4-4314-8aed-eb72904532df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:23:33.109535 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d824045-4df4-4314-8aed-eb72904532df service neutron] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:23:33.109818 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d824045-4df4-4314-8aed-eb72904532df service neutron] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:23:33.110089 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d824045-4df4-4314-8aed-eb72904532df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:23:33.120074 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d824045-4df4-4314-8aed-eb72904532df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:23:33.121607 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d824045-4df4-4314-8aed-eb72904532df service neutron] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:23:33.131601 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1403/2789] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:23:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:23:38.361326 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4f78d2-5b83-4e78-848d-a57094b789df None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:23:38.361815 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4f78d2-5b83-4e78-848d-a57094b789df None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:23:38.361815 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4f78d2-5b83-4e78-848d-a57094b789df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:23:38.394004 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-8b4f78d2-5b83-4e78-848d-a57094b789df None None] MFA Rules not processed for user `948d87bd5ba04663ac3fa8be3926d67a`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:23:38.395548 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-8b4f78d2-5b83-4e78-848d-a57094b789df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:23:38.490621 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1387/2790] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 09:23:38 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:23:38.495014 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d824045-4df4-4314-8aed-eb72904532df service neutron] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:23:38.505279 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d824045-4df4-4314-8aed-eb72904532df service neutron] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:23:38.511202 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a6f89a-67a5-4c00-ae6f-3b20c6b0e79b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:23:38.511818 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a6f89a-67a5-4c00-ae6f-3b20c6b0e79b admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:23:38.512095 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a6f89a-67a5-4c00-ae6f-3b20c6b0e79b admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:23:38.512365 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a6f89a-67a5-4c00-ae6f-3b20c6b0e79b admin admin] PATH_INFO: `/v3/projects` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:23:38.513020 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a6f89a-67a5-4c00-ae6f-3b20c6b0e79b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:23:38.515229 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a6f89a-67a5-4c00-ae6f-3b20c6b0e79b admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:23:38.627048 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1404/2791] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 09:23:38 2022] POST /identity/v3/projects => generated 385 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 09:23:38.631225 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4f78d2-5b83-4e78-848d-a57094b789df None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:23:38.643360 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4f78d2-5b83-4e78-848d-a57094b789df None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:23:38.652678 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:23:38.706462 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8350117-acec-4292-9575-c28a6854cc3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:23:38.712192 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8350117-acec-4292-9575-c28a6854cc3e admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:23:38.712192 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8350117-acec-4292-9575-c28a6854cc3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:23:38.712192 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8350117-acec-4292-9575-c28a6854cc3e admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:23:38.712192 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8350117-acec-4292-9575-c28a6854cc3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:23:38.712192 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8350117-acec-4292-9575-c28a6854cc3e admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:23:38.784039 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1388/2792] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 09:23:38 2022] POST /identity/v3/users => generated 299 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 09:23:38.785552 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a6f89a-67a5-4c00-ae6f-3b20c6b0e79b admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:23:38.797122 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a6f89a-67a5-4c00-ae6f-3b20c6b0e79b admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:23:38.802189 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcb785b-a974-44b4-b237-ccc21fcffbc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:23:38.802800 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcb785b-a974-44b4-b237-ccc21fcffbc2 admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:23:38.803144 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcb785b-a974-44b4-b237-ccc21fcffbc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:23:38.803338 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcb785b-a974-44b4-b237-ccc21fcffbc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:23:38.804011 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcb785b-a974-44b4-b237-ccc21fcffbc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:23:38.805707 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcb785b-a974-44b4-b237-ccc21fcffbc2 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:23:38.821016 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1405/2793] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 09:23:38 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 09:23:38.824460 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8350117-acec-4292-9575-c28a6854cc3e admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:23:38.836356 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8350117-acec-4292-9575-c28a6854cc3e admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:23:38.841220 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14705b86-19b4-448b-8177-fde404f8431f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:23:38.841873 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14705b86-19b4-448b-8177-fde404f8431f admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:23:38.842149 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14705b86-19b4-448b-8177-fde404f8431f admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:23:38.842417 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14705b86-19b4-448b-8177-fde404f8431f admin admin] PATH_INFO: `/v3/projects/8593f19e8bbd4afaa55a6c859b184e58/users/d8367dfbd2d141d5acf9ae3fa7f6b6ea/roles/ec3751692d9c4ecf81b4a71c936c46be` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:23:38.877705 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14705b86-19b4-448b-8177-fde404f8431f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8593f19e8bbd4afaa55a6c859b184e58, user_id=d8367dfbd2d141d5acf9ae3fa7f6b6ea, role_id=ec3751692d9c4ecf81b4a71c936c46be)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:23:38.880430 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14705b86-19b4-448b-8177-fde404f8431f admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:23:38.992746 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1389/2794] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 09:23:38 2022] PUT /identity/v3/projects/8593f19e8bbd4afaa55a6c859b184e58/users/d8367dfbd2d141d5acf9ae3fa7f6b6ea/roles/ec3751692d9c4ecf81b4a71c936c46be => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 09:23:38.997739 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcb785b-a974-44b4-b237-ccc21fcffbc2 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:23:39.010694 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcb785b-a974-44b4-b237-ccc21fcffbc2 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:23:39.073667 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02d907c-0af0-4559-93c4-e039f356e6ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:23:39.074325 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02d907c-0af0-4559-93c4-e039f356e6ec admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:23:39.074606 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02d907c-0af0-4559-93c4-e039f356e6ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:23:39.074878 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02d907c-0af0-4559-93c4-e039f356e6ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:23:39.075543 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02d907c-0af0-4559-93c4-e039f356e6ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:23:39.077159 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02d907c-0af0-4559-93c4-e039f356e6ec admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:23:39.091269 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1406/2795] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 09:23:38 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 09:23:39.092277 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14705b86-19b4-448b-8177-fde404f8431f admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:23:39.105565 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14705b86-19b4-448b-8177-fde404f8431f admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:23:39.110416 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d93fe9-90a9-4415-bf78-8cde4b27933a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:23:39.111116 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d93fe9-90a9-4415-bf78-8cde4b27933a admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:23:39.111398 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d93fe9-90a9-4415-bf78-8cde4b27933a admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:23:39.111674 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d93fe9-90a9-4415-bf78-8cde4b27933a admin admin] PATH_INFO: `/v3/projects/8593f19e8bbd4afaa55a6c859b184e58/users/d8367dfbd2d141d5acf9ae3fa7f6b6ea/roles/ec3751692d9c4ecf81b4a71c936c46be` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:23:39.115029 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d93fe9-90a9-4415-bf78-8cde4b27933a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8593f19e8bbd4afaa55a6c859b184e58, user_id=d8367dfbd2d141d5acf9ae3fa7f6b6ea, role_id=ec3751692d9c4ecf81b4a71c936c46be)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:23:39.116541 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d93fe9-90a9-4415-bf78-8cde4b27933a admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:23:39.128928 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1390/2796] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 09:23:39 2022] PUT /identity/v3/projects/8593f19e8bbd4afaa55a6c859b184e58/users/d8367dfbd2d141d5acf9ae3fa7f6b6ea/roles/ec3751692d9c4ecf81b4a71c936c46be => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:23:44.505824 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcb3ff0-c7dc-4023-be5a-c512f385695a None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:23:44.506332 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcb3ff0-c7dc-4023-be5a-c512f385695a None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:23:44.506603 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcb3ff0-c7dc-4023-be5a-c512f385695a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:23:44.551301 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-6dcb3ff0-c7dc-4023-be5a-c512f385695a None None] MFA Rules not processed for user `d8367dfbd2d141d5acf9ae3fa7f6b6ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:23:44.552703 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-6dcb3ff0-c7dc-4023-be5a-c512f385695a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:23:44.653010 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:23:44.658407 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1407/2797] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 09:23:44 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:23:44.678573 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d93fe9-90a9-4415-bf78-8cde4b27933a admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:23:44.680400 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-c2d93fe9-90a9-4415-bf78-8cde4b27933a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:23:44.745408 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d93fe9-90a9-4415-bf78-8cde4b27933a admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:23:44.749380 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bba8bb-9a7a-4227-adb1-cd86eb352ff0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3263af99fc2f48e79fd1e5b8a1e555a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:23:44.749983 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bba8bb-9a7a-4227-adb1-cd86eb352ff0 service glance] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:23:44.750258 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bba8bb-9a7a-4227-adb1-cd86eb352ff0 service glance] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:23:44.750525 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bba8bb-9a7a-4227-adb1-cd86eb352ff0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:23:44.811534 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bba8bb-9a7a-4227-adb1-cd86eb352ff0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:23:44.813026 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bba8bb-9a7a-4227-adb1-cd86eb352ff0 service glance] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:23:44.822430 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1391/2798] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:23:44 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:23:45.058378 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c75d4f-e238-440a-a2ba-4b2939f7e561 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:23:45.058821 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c75d4f-e238-440a-a2ba-4b2939f7e561 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:23:45.059237 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c75d4f-e238-440a-a2ba-4b2939f7e561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:23:45.135358 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-c1c75d4f-e238-440a-a2ba-4b2939f7e561 None None] MFA Rules not processed for user `c140d31bd2b84c48a170e3ca90a041ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:23:45.136609 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-c1c75d4f-e238-440a-a2ba-4b2939f7e561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:23:45.213182 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1408/2799] 10.222.0.57 () {58 vars in 1009 bytes} [Wed Aug 10 09:23:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:23:45.334496 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bba8bb-9a7a-4227-adb1-cd86eb352ff0 service glance] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:23:45.335986 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-d5bba8bb-9a7a-4227-adb1-cd86eb352ff0 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:23:45.367298 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:23:45.443018 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bba8bb-9a7a-4227-adb1-cd86eb352ff0 service glance] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:23:45.448784 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fd4742-0ed1-42b1-8c51-3305d37ee12d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4b26ba3a9e04c38b52db2a46198a53c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:23:45.449427 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fd4742-0ed1-42b1-8c51-3305d37ee12d service swift] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:23:45.449709 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fd4742-0ed1-42b1-8c51-3305d37ee12d service swift] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:23:45.449981 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fd4742-0ed1-42b1-8c51-3305d37ee12d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:23:45.458994 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fd4742-0ed1-42b1-8c51-3305d37ee12d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:23:45.460723 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fd4742-0ed1-42b1-8c51-3305d37ee12d service swift] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:23:45.470253 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1392/2800] 10.222.0.57 () {60 vars in 1384 bytes} [Wed Aug 10 09:23:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 09:23:55.208414 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=45 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:23:55.391101 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e299824-4e57-42fe-8a67-a41dfb3df98e None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:23:55.391386 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e299824-4e57-42fe-8a67-a41dfb3df98e None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:23:55.391662 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e299824-4e57-42fe-8a67-a41dfb3df98e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:23:55.411352 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-3e299824-4e57-42fe-8a67-a41dfb3df98e None None] MFA Rules not processed for user `948d87bd5ba04663ac3fa8be3926d67a`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:23:55.412539 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-3e299824-4e57-42fe-8a67-a41dfb3df98e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:23:55.420573 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1409/2801] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 09:23:55 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:23:55.466645 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=33 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:23:59.001487 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189fbd56-0154-48c2-9f86-538a7d3ab45e None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:23:59.002119 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189fbd56-0154-48c2-9f86-538a7d3ab45e None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:23:59.002119 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189fbd56-0154-48c2-9f86-538a7d3ab45e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:23:59.022916 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-189fbd56-0154-48c2-9f86-538a7d3ab45e None None] MFA Rules not processed for user `c140d31bd2b84c48a170e3ca90a041ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:23:59.023845 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-189fbd56-0154-48c2-9f86-538a7d3ab45e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:23:59.032147 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1393/2802] 10.222.0.57 () {58 vars in 1009 bytes} [Wed Aug 10 09:23:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:23:59.045263 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e299824-4e57-42fe-8a67-a41dfb3df98e None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:23:59.057546 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e299824-4e57-42fe-8a67-a41dfb3df98e None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:23:59.062802 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9090e4-d0f7-4231-9f2c-cf1cfaa3d598 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4b26ba3a9e04c38b52db2a46198a53c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:23:59.063511 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9090e4-d0f7-4231-9f2c-cf1cfaa3d598 service swift] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:23:59.063793 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9090e4-d0f7-4231-9f2c-cf1cfaa3d598 service swift] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:23:59.064092 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9090e4-d0f7-4231-9f2c-cf1cfaa3d598 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:23:59.072430 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9090e4-d0f7-4231-9f2c-cf1cfaa3d598 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:23:59.074028 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9090e4-d0f7-4231-9f2c-cf1cfaa3d598 service swift] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:23:59.081895 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1410/2803] 10.222.0.57 () {60 vars in 1384 bytes} [Wed Aug 10 09:23:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 09:24:09.016430 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=3 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:24:09.078664 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=9 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:24:34.096425 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189fbd56-0154-48c2-9f86-538a7d3ab45e None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:24:34.150896 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189fbd56-0154-48c2-9f86-538a7d3ab45e None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:24:34.154699 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663e08a0-b70d-482f-ae20-91cba869ec3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:24:34.155712 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663e08a0-b70d-482f-ae20-91cba869ec3d service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:24:34.155712 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663e08a0-b70d-482f-ae20-91cba869ec3d service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:24:34.155895 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663e08a0-b70d-482f-ae20-91cba869ec3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:24:34.164868 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663e08a0-b70d-482f-ae20-91cba869ec3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:24:34.166304 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663e08a0-b70d-482f-ae20-91cba869ec3d service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:24:34.176708 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1394/2804] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:24:34 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:24:34.844616 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfa725d-9ce3-4843-a50c-d6be5ea1bed6 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:24:34.845132 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfa725d-9ce3-4843-a50c-d6be5ea1bed6 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:24:34.845209 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfa725d-9ce3-4843-a50c-d6be5ea1bed6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:24:34.864538 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-0bfa725d-9ce3-4843-a50c-d6be5ea1bed6 None None] MFA Rules not processed for user `d113e0f5215c44699b8eb744ad712735`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:24:34.865828 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-0bfa725d-9ce3-4843-a50c-d6be5ea1bed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:24:34.926287 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1411/2805] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 09:24:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:24:44.172620 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=26 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:24:44.918434 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=25 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:24:53.893348 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663e08a0-b70d-482f-ae20-91cba869ec3d service neutron] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:24:53.894847 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-663e08a0-b70d-482f-ae20-91cba869ec3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:24:53.967913 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663e08a0-b70d-482f-ae20-91cba869ec3d service neutron] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:24:53.973439 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda2983c-d639-4233-9b04-7d543917df16 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a943008193914f76a663d489bd73e873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:24:53.974028 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda2983c-d639-4233-9b04-7d543917df16 service nova] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:24:53.974304 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda2983c-d639-4233-9b04-7d543917df16 service nova] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:24:53.974571 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda2983c-d639-4233-9b04-7d543917df16 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:24:53.976291 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-fda2983c-d639-4233-9b04-7d543917df16 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:24:53.990969 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda2983c-d639-4233-9b04-7d543917df16 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:24:53.992540 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda2983c-d639-4233-9b04-7d543917df16 service nova] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:24:54.003959 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1395/2806] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:24:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:25:02.612134 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c57d18f-26d2-48a5-b7a1-91fe49a19431 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:25:02.612602 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c57d18f-26d2-48a5-b7a1-91fe49a19431 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:25:02.612744 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c57d18f-26d2-48a5-b7a1-91fe49a19431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:25:02.632515 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-2c57d18f-26d2-48a5-b7a1-91fe49a19431 None None] MFA Rules not processed for user `c140d31bd2b84c48a170e3ca90a041ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:25:02.633820 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-2c57d18f-26d2-48a5-b7a1-91fe49a19431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:25:02.642151 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1412/2807] 10.222.0.57 () {58 vars in 1009 bytes} [Wed Aug 10 09:25:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:25:02.682063 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda2983c-d639-4233-9b04-7d543917df16 service nova] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:25:02.694318 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda2983c-d639-4233-9b04-7d543917df16 service nova] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:25:02.699436 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998ec672-5d9b-4d0e-85c6-362f71bbb59c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4b26ba3a9e04c38b52db2a46198a53c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:25:02.700054 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998ec672-5d9b-4d0e-85c6-362f71bbb59c service swift] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:25:02.700328 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998ec672-5d9b-4d0e-85c6-362f71bbb59c service swift] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:25:02.700595 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998ec672-5d9b-4d0e-85c6-362f71bbb59c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:25:02.709457 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998ec672-5d9b-4d0e-85c6-362f71bbb59c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:25:02.711015 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998ec672-5d9b-4d0e-85c6-362f71bbb59c service swift] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:25:02.718800 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1396/2808] 10.222.0.57 () {60 vars in 1384 bytes} [Wed Aug 10 09:25:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 10 09:25:12.627278 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=3 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:25:12.715670 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=40 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:25:18.998177 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615f6d4b-fbcc-407b-a1e1-e7aaf51e58e0 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:25:18.998682 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615f6d4b-fbcc-407b-a1e1-e7aaf51e58e0 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:25:18.998770 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615f6d4b-fbcc-407b-a1e1-e7aaf51e58e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:25:19.018731 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-615f6d4b-fbcc-407b-a1e1-e7aaf51e58e0 None None] MFA Rules not processed for user `c140d31bd2b84c48a170e3ca90a041ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:25:19.019826 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-615f6d4b-fbcc-407b-a1e1-e7aaf51e58e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:25:19.026840 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1413/2809] 10.222.0.57 () {58 vars in 967 bytes} [Wed Aug 10 09:25:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:25:19.040237 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998ec672-5d9b-4d0e-85c6-362f71bbb59c service swift] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:25:19.051290 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998ec672-5d9b-4d0e-85c6-362f71bbb59c service swift] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:25:19.056722 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad5e6ba-9ab9-45cf-9174-ec689018d8d5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4b26ba3a9e04c38b52db2a46198a53c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:25:19.063270 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad5e6ba-9ab9-45cf-9174-ec689018d8d5 service swift] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:25:19.063270 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad5e6ba-9ab9-45cf-9174-ec689018d8d5 service swift] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:25:19.063270 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad5e6ba-9ab9-45cf-9174-ec689018d8d5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:25:19.066580 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad5e6ba-9ab9-45cf-9174-ec689018d8d5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:25:19.068197 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad5e6ba-9ab9-45cf-9174-ec689018d8d5 service swift] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:25:19.076226 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1397/2810] 10.222.0.57 () {60 vars in 1384 bytes} [Wed Aug 10 09:25:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 09:25:21.955936 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615f6d4b-fbcc-407b-a1e1-e7aaf51e58e0 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:25:21.957555 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-615f6d4b-fbcc-407b-a1e1-e7aaf51e58e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:25:22.025316 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615f6d4b-fbcc-407b-a1e1-e7aaf51e58e0 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:25:22.029719 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fdbceb-d0b3-4fcb-b0b4-155bd3b19f64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:25:22.030335 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fdbceb-d0b3-4fcb-b0b4-155bd3b19f64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:25:22.030609 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fdbceb-d0b3-4fcb-b0b4-155bd3b19f64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:25:22.030876 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fdbceb-d0b3-4fcb-b0b4-155bd3b19f64 admin admin] PATH_INFO: `/v3/users/d113e0f5215c44699b8eb744ad712735` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:25:22.032014 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fdbceb-d0b3-4fcb-b0b4-155bd3b19f64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d113e0f5215c44699b8eb744ad712735)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:25:22.034140 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fdbceb-d0b3-4fcb-b0b4-155bd3b19f64 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:25:22.236509 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-86fdbceb-d0b3-4fcb-b0b4-155bd3b19f64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd113e0f5215c44699b8eb744ad712735'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:25:22.247981 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-86fdbceb-d0b3-4fcb-b0b4-155bd3b19f64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd113e0f5215c44699b8eb744ad712735'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:25:22.255306 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-86fdbceb-d0b3-4fcb-b0b4-155bd3b19f64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd113e0f5215c44699b8eb744ad712735'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:25:22.324582 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-86fdbceb-d0b3-4fcb-b0b4-155bd3b19f64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd113e0f5215c44699b8eb744ad712735'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:25:22.325222 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-86fdbceb-d0b3-4fcb-b0b4-155bd3b19f64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd113e0f5215c44699b8eb744ad712735'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:25:22.331983 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1414/2811] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 09:25:21 2022] DELETE /identity/v3/users/d113e0f5215c44699b8eb744ad712735 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:25:22.562553 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad5e6ba-9ab9-45cf-9174-ec689018d8d5 service swift] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:25:22.576292 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad5e6ba-9ab9-45cf-9174-ec689018d8d5 service swift] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:25:22.633232 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7488a4-0f95-44c8-91e8-f9dfa1653876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:25:22.633850 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7488a4-0f95-44c8-91e8-f9dfa1653876 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:25:22.634129 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7488a4-0f95-44c8-91e8-f9dfa1653876 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:25:22.634393 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7488a4-0f95-44c8-91e8-f9dfa1653876 admin admin] PATH_INFO: `/v3/projects/e1616e4f3f89412f9a3c631208ecb493` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:25:22.635485 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7488a4-0f95-44c8-91e8-f9dfa1653876 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1616e4f3f89412f9a3c631208ecb493)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:25:22.637400 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7488a4-0f95-44c8-91e8-f9dfa1653876 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:25:22.744715 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-7e7488a4-0f95-44c8-91e8-f9dfa1653876 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1616e4f3f89412f9a3c631208ecb493', 'request_id': 'req-7e7488a4-0f95-44c8-91e8-f9dfa1653876'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:25:22.827665 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-7e7488a4-0f95-44c8-91e8-f9dfa1653876 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1616e4f3f89412f9a3c631208ecb493', 'request_id': 'req-7e7488a4-0f95-44c8-91e8-f9dfa1653876'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:25:22.834297 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-7e7488a4-0f95-44c8-91e8-f9dfa1653876 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1616e4f3f89412f9a3c631208ecb493', 'request_id': 'req-7e7488a4-0f95-44c8-91e8-f9dfa1653876'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:25:22.845239 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-7e7488a4-0f95-44c8-91e8-f9dfa1653876 admin admin] The token cache is being invalidate because project e1616e4f3f89412f9a3c631208ecb493 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 09:25:22.846019 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-7e7488a4-0f95-44c8-91e8-f9dfa1653876 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:25:22.846644 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-7e7488a4-0f95-44c8-91e8-f9dfa1653876 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:25:22.848105 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1398/2812] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 09:25:22 2022] DELETE /identity/v3/projects/e1616e4f3f89412f9a3c631208ecb493 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:25:22.852983 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8087cf-d247-47f7-b654-003b88c959e1 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:25:22.853270 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8087cf-d247-47f7-b654-003b88c959e1 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:25:22.853544 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8087cf-d247-47f7-b654-003b88c959e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:25:22.873877 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-9d8087cf-d247-47f7-b654-003b88c959e1 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:25:22.875073 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-9d8087cf-d247-47f7-b654-003b88c959e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:25:22.934305 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1415/2813] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 09:25:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 09:25:22.941898 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f02811-74d4-415b-90e0-2f0cf5d449be None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:25:22.942177 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f02811-74d4-415b-90e0-2f0cf5d449be None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:25:22.942473 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f02811-74d4-415b-90e0-2f0cf5d449be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:25:22.961636 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-00f02811-74d4-415b-90e0-2f0cf5d449be None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:25:22.962815 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-00f02811-74d4-415b-90e0-2f0cf5d449be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:25:22.970969 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1399/2814] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 09:25:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 09:25:22.975066 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8087cf-d247-47f7-b654-003b88c959e1 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:25:22.984497 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8087cf-d247-47f7-b654-003b88c959e1 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:25:22.989786 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcc6e58-5a60-494f-9819-deda3c04a6fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:25:22.990374 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcc6e58-5a60-494f-9819-deda3c04a6fc admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:25:22.990649 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcc6e58-5a60-494f-9819-deda3c04a6fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:25:22.990916 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcc6e58-5a60-494f-9819-deda3c04a6fc admin admin] PATH_INFO: `/v3/domains` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:25:22.991553 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcc6e58-5a60-494f-9819-deda3c04a6fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:25:22.991852 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcc6e58-5a60-494f-9819-deda3c04a6fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:25:22.993468 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcc6e58-5a60-494f-9819-deda3c04a6fc admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:25:23.001825 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:25:23.005312 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1416/2815] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Aug 10 09:25:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:25:23.014518 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f02811-74d4-415b-90e0-2f0cf5d449be None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:25:23.023889 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f02811-74d4-415b-90e0-2f0cf5d449be None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:25:23.028464 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58d2c88-efe3-4243-8d3a-16703fdce3ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:25:23.029091 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58d2c88-efe3-4243-8d3a-16703fdce3ca admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:25:23.029320 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58d2c88-efe3-4243-8d3a-16703fdce3ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:25:23.029588 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58d2c88-efe3-4243-8d3a-16703fdce3ca admin admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:25:23.030199 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58d2c88-efe3-4243-8d3a-16703fdce3ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:25:23.032092 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58d2c88-efe3-4243-8d3a-16703fdce3ca admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:25:23.120612 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1400/2816] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 09:25:23 2022] POST /identity/v3/projects => generated 381 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 09:25:23.124745 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcc6e58-5a60-494f-9819-deda3c04a6fc admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:25:23.134456 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcc6e58-5a60-494f-9819-deda3c04a6fc admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:25:23.190510 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f218b552-57d8-4b41-bc93-9feb74bd207b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:25:23.191112 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f218b552-57d8-4b41-bc93-9feb74bd207b admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:25:23.191394 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f218b552-57d8-4b41-bc93-9feb74bd207b admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:25:23.191664 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f218b552-57d8-4b41-bc93-9feb74bd207b admin admin] PATH_INFO: `/v3/users` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:25:23.192292 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f218b552-57d8-4b41-bc93-9feb74bd207b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:25:23.194239 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f218b552-57d8-4b41-bc93-9feb74bd207b admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:25:23.263732 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1417/2817] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 09:25:23 2022] POST /identity/v3/users => generated 291 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 09:25:23.268263 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58d2c88-efe3-4243-8d3a-16703fdce3ca admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:25:23.278610 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58d2c88-efe3-4243-8d3a-16703fdce3ca admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:25:23.283873 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e8f4ba-a917-4bf8-813d-171a70041453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:25:23.284497 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e8f4ba-a917-4bf8-813d-171a70041453 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:25:23.284773 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e8f4ba-a917-4bf8-813d-171a70041453 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:25:23.285081 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e8f4ba-a917-4bf8-813d-171a70041453 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:25:23.285715 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e8f4ba-a917-4bf8-813d-171a70041453 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:25:23.287291 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e8f4ba-a917-4bf8-813d-171a70041453 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:25:23.297013 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1401/2818] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 09:25:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 09:25:23.301448 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f218b552-57d8-4b41-bc93-9feb74bd207b admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:25:23.311490 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f218b552-57d8-4b41-bc93-9feb74bd207b admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:25:23.316297 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4b272a-0eb2-4374-84e2-994bb3714841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:25:23.316938 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4b272a-0eb2-4374-84e2-994bb3714841 admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:25:23.317237 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4b272a-0eb2-4374-84e2-994bb3714841 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:25:23.317508 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4b272a-0eb2-4374-84e2-994bb3714841 admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:25:23.318144 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4b272a-0eb2-4374-84e2-994bb3714841 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:25:23.319739 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4b272a-0eb2-4374-84e2-994bb3714841 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:25:23.330650 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1418/2819] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 09:25:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 09:25:23.334602 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e8f4ba-a917-4bf8-813d-171a70041453 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:25:23.344591 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e8f4ba-a917-4bf8-813d-171a70041453 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:25:23.350064 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3af92d-bf3a-4e92-98ae-776b6461f043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:25:23.350695 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3af92d-bf3a-4e92-98ae-776b6461f043 admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:25:23.350973 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3af92d-bf3a-4e92-98ae-776b6461f043 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:25:23.351280 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3af92d-bf3a-4e92-98ae-776b6461f043 admin admin] PATH_INFO: `/v3/projects/25372be0a838467aab413517420d3164/users/3d176cb1bc66423b8647a5a76efc9344/roles/44b64db484f24a46bf0b7ced46b519bf` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:25:23.365995 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3af92d-bf3a-4e92-98ae-776b6461f043 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25372be0a838467aab413517420d3164, user_id=3d176cb1bc66423b8647a5a76efc9344, role_id=44b64db484f24a46bf0b7ced46b519bf)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:25:23.368672 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3af92d-bf3a-4e92-98ae-776b6461f043 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:25:23.442567 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1402/2820] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 09:25:23 2022] PUT /identity/v3/projects/25372be0a838467aab413517420d3164/users/3d176cb1bc66423b8647a5a76efc9344/roles/44b64db484f24a46bf0b7ced46b519bf => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:25:23.447512 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5cf64e-5d0f-40d8-a92e-7211544a17b5 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:25:23.447791 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5cf64e-5d0f-40d8-a92e-7211544a17b5 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:25:23.448056 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5cf64e-5d0f-40d8-a92e-7211544a17b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:25:23.468646 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-fd5cf64e-5d0f-40d8-a92e-7211544a17b5 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:25:23.469902 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-fd5cf64e-5d0f-40d8-a92e-7211544a17b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:25:23.528509 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1419/2821] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 09:25:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:25:23.540105 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3af92d-bf3a-4e92-98ae-776b6461f043 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:25:23.541795 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-4e3af92d-bf3a-4e92-98ae-776b6461f043 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:25:23.559681 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:25:23.609778 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3af92d-bf3a-4e92-98ae-776b6461f043 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:25:23.614380 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c87274-bd3b-4e93-872d-3aa3c47bca7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:25:23.614977 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c87274-bd3b-4e93-872d-3aa3c47bca7c service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:25:23.615252 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c87274-bd3b-4e93-872d-3aa3c47bca7c service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:25:23.615520 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c87274-bd3b-4e93-872d-3aa3c47bca7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:25:23.624505 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c87274-bd3b-4e93-872d-3aa3c47bca7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:25:23.625945 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c87274-bd3b-4e93-872d-3aa3c47bca7c service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:25:23.635865 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1403/2822] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:25:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:25:24.760326 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fa27fc-f3c1-4167-8255-c34e04075a29 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:25:24.764045 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fa27fc-f3c1-4167-8255-c34e04075a29 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:25:24.764045 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fa27fc-f3c1-4167-8255-c34e04075a29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:25:24.789734 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-23fa27fc-f3c1-4167-8255-c34e04075a29 None None] MFA Rules not processed for user `3d176cb1bc66423b8647a5a76efc9344`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:25:24.790960 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-23fa27fc-f3c1-4167-8255-c34e04075a29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:25:24.854919 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1420/2823] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 09:25:24 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:25:24.866664 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c87274-bd3b-4e93-872d-3aa3c47bca7c service neutron] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:25:24.868104 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-f8c87274-bd3b-4e93-872d-3aa3c47bca7c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:25:24.936730 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c87274-bd3b-4e93-872d-3aa3c47bca7c service neutron] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:25:24.940865 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6428c5-c6bc-43be-9470-de39f33adf90 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ba74e28ced4cbba857f54222dfbe41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:25:24.941577 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6428c5-c6bc-43be-9470-de39f33adf90 service cinder] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:25:24.941859 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6428c5-c6bc-43be-9470-de39f33adf90 service cinder] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:25:24.942126 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6428c5-c6bc-43be-9470-de39f33adf90 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:25:24.950223 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6428c5-c6bc-43be-9470-de39f33adf90 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:25:24.951657 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6428c5-c6bc-43be-9470-de39f33adf90 service cinder] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:25:24.960533 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1404/2824] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:25:24 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:25:26.615176 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e04010b-43e5-4ffa-86d4-fca9677c416c None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:25:26.615678 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e04010b-43e5-4ffa-86d4-fca9677c416c None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:25:26.615770 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e04010b-43e5-4ffa-86d4-fca9677c416c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:25:26.634594 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-3e04010b-43e5-4ffa-86d4-fca9677c416c None None] MFA Rules not processed for user `3d176cb1bc66423b8647a5a76efc9344`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:25:26.635796 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-3e04010b-43e5-4ffa-86d4-fca9677c416c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:25:26.642691 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1421/2825] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 09:25:26 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:25:34.956587 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=56 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:25:36.630018 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=96,keystone:INSERT=3 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:26:10.323885 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6428c5-c6bc-43be-9470-de39f33adf90 service cinder] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:26:10.325168 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-7d6428c5-c6bc-43be-9470-de39f33adf90 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:26:10.415392 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6428c5-c6bc-43be-9470-de39f33adf90 service cinder] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:26:10.420725 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1bcdf5-4690-4c01-ac22-44d7d56fe200 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2912307c56594125b157f1cfdf7c96a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:26:10.421410 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1bcdf5-4690-4c01-ac22-44d7d56fe200 service placement] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:26:10.421690 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1bcdf5-4690-4c01-ac22-44d7d56fe200 service placement] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:26:10.421961 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1bcdf5-4690-4c01-ac22-44d7d56fe200 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:26:10.423853 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-fd1bcdf5-4690-4c01-ac22-44d7d56fe200 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:26:10.441125 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1bcdf5-4690-4c01-ac22-44d7d56fe200 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:26:10.442728 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1bcdf5-4690-4c01-ac22-44d7d56fe200 service placement] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:26:10.453833 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1405/2826] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:26:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:26:20.449435 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=37 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:26:57.576382 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51504b7-c445-4165-a2a5-f1f4280d510c None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:26:57.576888 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51504b7-c445-4165-a2a5-f1f4280d510c None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:26:57.577035 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51504b7-c445-4165-a2a5-f1f4280d510c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:26:57.596015 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-c51504b7-c445-4165-a2a5-f1f4280d510c None None] MFA Rules not processed for user `3d176cb1bc66423b8647a5a76efc9344`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:26:57.597236 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-c51504b7-c445-4165-a2a5-f1f4280d510c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:26:57.604002 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1422/2827] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 09:26:57 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:27:07.590244 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=3 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:27:38.424596 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1bcdf5-4690-4c01-ac22-44d7d56fe200 service placement] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:27:38.426215 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-fd1bcdf5-4690-4c01-ac22-44d7d56fe200 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:27:38.438388 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1bcdf5-4690-4c01-ac22-44d7d56fe200 service placement] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:27:38.443715 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f13a254-d38a-4ca8-9749-b7a1ce9183e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ba74e28ced4cbba857f54222dfbe41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:27:38.444321 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f13a254-d38a-4ca8-9749-b7a1ce9183e0 service cinder] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:27:38.444604 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f13a254-d38a-4ca8-9749-b7a1ce9183e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:27:38.451606 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f13a254-d38a-4ca8-9749-b7a1ce9183e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:27:38.451606 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-2f13a254-d38a-4ca8-9749-b7a1ce9183e0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:27:38.531472 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f13a254-d38a-4ca8-9749-b7a1ce9183e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:27:38.532892 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f13a254-d38a-4ca8-9749-b7a1ce9183e0 service cinder] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:27:38.551688 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1406/2828] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:27:38 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:27:42.558348 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51504b7-c445-4165-a2a5-f1f4280d510c None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:27:42.559748 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-c51504b7-c445-4165-a2a5-f1f4280d510c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:27:42.572342 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51504b7-c445-4165-a2a5-f1f4280d510c None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:27:42.575873 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46eb927-9893-496e-87bb-f7ac1e340019 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:27:42.576465 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46eb927-9893-496e-87bb-f7ac1e340019 service neutron] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:27:42.576747 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46eb927-9893-496e-87bb-f7ac1e340019 service neutron] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:27:42.577060 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46eb927-9893-496e-87bb-f7ac1e340019 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:27:42.578718 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-d46eb927-9893-496e-87bb-f7ac1e340019 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:27:42.613457 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46eb927-9893-496e-87bb-f7ac1e340019 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:27:42.614898 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46eb927-9893-496e-87bb-f7ac1e340019 service neutron] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:27:42.634502 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1423/2829] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:27:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:27:44.453395 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1762fc1b-98f8-412b-9c42-e51279c58faf None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:27:44.454359 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1762fc1b-98f8-412b-9c42-e51279c58faf None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:27:44.455286 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1762fc1b-98f8-412b-9c42-e51279c58faf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:27:44.482367 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-1762fc1b-98f8-412b-9c42-e51279c58faf None None] MFA Rules not processed for user `c140d31bd2b84c48a170e3ca90a041ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:27:44.484142 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-1762fc1b-98f8-412b-9c42-e51279c58faf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:27:44.553924 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1407/2830] 10.222.0.57 () {58 vars in 967 bytes} [Wed Aug 10 09:27:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:27:44.698545 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46eb927-9893-496e-87bb-f7ac1e340019 service neutron] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:27:44.698545 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-d46eb927-9893-496e-87bb-f7ac1e340019 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:27:44.799123 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46eb927-9893-496e-87bb-f7ac1e340019 service neutron] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:27:44.804465 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dc24f5-5c10-4bd7-9ee0-43bf715a7fad service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4b26ba3a9e04c38b52db2a46198a53c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:27:44.805036 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dc24f5-5c10-4bd7-9ee0-43bf715a7fad service swift] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:27:44.805318 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dc24f5-5c10-4bd7-9ee0-43bf715a7fad service swift] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:27:44.805592 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dc24f5-5c10-4bd7-9ee0-43bf715a7fad service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:27:44.814724 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dc24f5-5c10-4bd7-9ee0-43bf715a7fad service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:27:44.816998 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dc24f5-5c10-4bd7-9ee0-43bf715a7fad service swift] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:27:44.825869 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1424/2831] 10.222.0.57 () {60 vars in 1384 bytes} [Wed Aug 10 09:27:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 09:27:46.697747 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1762fc1b-98f8-412b-9c42-e51279c58faf None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:27:46.699296 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-1762fc1b-98f8-412b-9c42-e51279c58faf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:27:46.717726 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1762fc1b-98f8-412b-9c42-e51279c58faf None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:27:46.724368 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedfef17-5f71-4feb-ab13-26f59d7a8d3d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2912307c56594125b157f1cfdf7c96a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:27:46.725155 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedfef17-5f71-4feb-ab13-26f59d7a8d3d service placement] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:27:46.725775 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedfef17-5f71-4feb-ab13-26f59d7a8d3d service placement] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:27:46.726242 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedfef17-5f71-4feb-ab13-26f59d7a8d3d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:27:46.728103 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-bedfef17-5f71-4feb-ab13-26f59d7a8d3d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:27:46.741438 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedfef17-5f71-4feb-ab13-26f59d7a8d3d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:27:46.743432 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedfef17-5f71-4feb-ab13-26f59d7a8d3d service placement] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:27:46.753568 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1408/2832] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:27:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:27:46.885263 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dc24f5-5c10-4bd7-9ee0-43bf715a7fad service swift] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:27:46.885263 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-97dc24f5-5c10-4bd7-9ee0-43bf715a7fad service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:27:46.899079 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dc24f5-5c10-4bd7-9ee0-43bf715a7fad service swift] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:27:46.903742 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017930fd-0f47-44c8-974f-93898a8272cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:27:46.904373 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017930fd-0f47-44c8-974f-93898a8272cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:27:46.904679 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017930fd-0f47-44c8-974f-93898a8272cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:27:46.904980 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017930fd-0f47-44c8-974f-93898a8272cc admin admin] PATH_INFO: `/v3/users/9797df3e05944552b741f53773abf80a` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:27:46.906122 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017930fd-0f47-44c8-974f-93898a8272cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=9797df3e05944552b741f53773abf80a)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:27:46.908093 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017930fd-0f47-44c8-974f-93898a8272cc admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:27:47.109906 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-017930fd-0f47-44c8-974f-93898a8272cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9797df3e05944552b741f53773abf80a'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:27:47.132231 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-017930fd-0f47-44c8-974f-93898a8272cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9797df3e05944552b741f53773abf80a'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:27:47.142573 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-017930fd-0f47-44c8-974f-93898a8272cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9797df3e05944552b741f53773abf80a'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:27:47.189805 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-017930fd-0f47-44c8-974f-93898a8272cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9797df3e05944552b741f53773abf80a'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:27:47.190373 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-017930fd-0f47-44c8-974f-93898a8272cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9797df3e05944552b741f53773abf80a'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:27:47.196826 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1425/2833] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 09:27:46 2022] DELETE /identity/v3/users/9797df3e05944552b741f53773abf80a => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:27:47.450388 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedfef17-5f71-4feb-ab13-26f59d7a8d3d service placement] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:27:47.459234 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedfef17-5f71-4feb-ab13-26f59d7a8d3d service placement] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:27:47.553399 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e15a122-7d2c-4ad7-b2d7-01d4d6c98e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:27:47.554068 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e15a122-7d2c-4ad7-b2d7-01d4d6c98e6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:27:47.555111 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e15a122-7d2c-4ad7-b2d7-01d4d6c98e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:27:47.555407 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e15a122-7d2c-4ad7-b2d7-01d4d6c98e6f admin admin] PATH_INFO: `/v3/projects/95db0a5b805944418265a85dc6cef4fc` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:27:47.557077 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e15a122-7d2c-4ad7-b2d7-01d4d6c98e6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=95db0a5b805944418265a85dc6cef4fc)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:27:47.559849 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e15a122-7d2c-4ad7-b2d7-01d4d6c98e6f admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:27:47.563363 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:27:47.680880 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-7e15a122-7d2c-4ad7-b2d7-01d4d6c98e6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95db0a5b805944418265a85dc6cef4fc', 'request_id': 'req-7e15a122-7d2c-4ad7-b2d7-01d4d6c98e6f'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:27:47.770899 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-7e15a122-7d2c-4ad7-b2d7-01d4d6c98e6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95db0a5b805944418265a85dc6cef4fc', 'request_id': 'req-7e15a122-7d2c-4ad7-b2d7-01d4d6c98e6f'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:27:47.778678 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-7e15a122-7d2c-4ad7-b2d7-01d4d6c98e6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95db0a5b805944418265a85dc6cef4fc', 'request_id': 'req-7e15a122-7d2c-4ad7-b2d7-01d4d6c98e6f'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:27:47.792083 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-7e15a122-7d2c-4ad7-b2d7-01d4d6c98e6f admin admin] The token cache is being invalidate because project 95db0a5b805944418265a85dc6cef4fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 09:27:47.792512 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-7e15a122-7d2c-4ad7-b2d7-01d4d6c98e6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:27:47.793468 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-7e15a122-7d2c-4ad7-b2d7-01d4d6c98e6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:27:47.795104 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1409/2834] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 09:27:47 2022] DELETE /identity/v3/projects/95db0a5b805944418265a85dc6cef4fc => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:27:47.803262 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba5fb30-acbc-401b-9670-1fb9665d824c None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:27:47.803546 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba5fb30-acbc-401b-9670-1fb9665d824c None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:27:47.803880 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba5fb30-acbc-401b-9670-1fb9665d824c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:27:47.835379 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:27:47.882505 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-8ba5fb30-acbc-401b-9670-1fb9665d824c None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:27:47.883794 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-8ba5fb30-acbc-401b-9670-1fb9665d824c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:27:47.964178 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1426/2835] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 09:27:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:27:47.965260 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d00e0c7-2bf5-4a14-9707-35c67535f297 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:27:47.965260 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d00e0c7-2bf5-4a14-9707-35c67535f297 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:27:47.965260 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d00e0c7-2bf5-4a14-9707-35c67535f297 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:27:47.997307 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-2d00e0c7-2bf5-4a14-9707-35c67535f297 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:27:47.998722 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-2d00e0c7-2bf5-4a14-9707-35c67535f297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:27:48.012569 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1410/2836] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 09:27:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 09:27:48.018797 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba5fb30-acbc-401b-9670-1fb9665d824c None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:27:48.029689 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba5fb30-acbc-401b-9670-1fb9665d824c None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:27:48.034548 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55318a0-da53-446a-90c6-b49e249c6995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:27:48.035884 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55318a0-da53-446a-90c6-b49e249c6995 admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:27:48.036228 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55318a0-da53-446a-90c6-b49e249c6995 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:27:48.036755 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55318a0-da53-446a-90c6-b49e249c6995 admin admin] PATH_INFO: `/v3/domains` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:27:48.037523 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55318a0-da53-446a-90c6-b49e249c6995 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:27:48.038015 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55318a0-da53-446a-90c6-b49e249c6995 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:27:48.040074 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55318a0-da53-446a-90c6-b49e249c6995 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:27:48.053339 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1427/2837] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Aug 10 09:27:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:27:48.058288 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d00e0c7-2bf5-4a14-9707-35c67535f297 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:27:48.068678 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d00e0c7-2bf5-4a14-9707-35c67535f297 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:27:48.074153 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae52a5c-5937-4c1e-a603-5753459f024c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:27:48.074769 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae52a5c-5937-4c1e-a603-5753459f024c admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:27:48.075051 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae52a5c-5937-4c1e-a603-5753459f024c admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:27:48.075322 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae52a5c-5937-4c1e-a603-5753459f024c admin admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:27:48.075958 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae52a5c-5937-4c1e-a603-5753459f024c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:27:48.077934 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae52a5c-5937-4c1e-a603-5753459f024c admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:27:48.218690 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1411/2838] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 09:27:48 2022] POST /identity/v3/projects => generated 367 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 09:27:48.223094 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55318a0-da53-446a-90c6-b49e249c6995 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:27:48.234164 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55318a0-da53-446a-90c6-b49e249c6995 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:27:48.294129 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db25b6d9-fe53-44d3-a9fc-f3796c709140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:27:48.294743 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db25b6d9-fe53-44d3-a9fc-f3796c709140 admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:27:48.295027 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db25b6d9-fe53-44d3-a9fc-f3796c709140 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:27:48.295297 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db25b6d9-fe53-44d3-a9fc-f3796c709140 admin admin] PATH_INFO: `/v3/users` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:27:48.295954 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db25b6d9-fe53-44d3-a9fc-f3796c709140 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:27:48.298021 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db25b6d9-fe53-44d3-a9fc-f3796c709140 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:27:48.390384 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1428/2839] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 09:27:48 2022] POST /identity/v3/users => generated 284 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 09:27:48.395884 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae52a5c-5937-4c1e-a603-5753459f024c admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:27:48.406284 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae52a5c-5937-4c1e-a603-5753459f024c admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:27:48.412066 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1d8d46-ec94-48be-b823-70288f67a5b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:27:48.412693 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1d8d46-ec94-48be-b823-70288f67a5b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:27:48.412998 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1d8d46-ec94-48be-b823-70288f67a5b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:27:48.413314 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1d8d46-ec94-48be-b823-70288f67a5b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:27:48.413944 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1d8d46-ec94-48be-b823-70288f67a5b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:27:48.415544 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1d8d46-ec94-48be-b823-70288f67a5b4 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:27:48.426452 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1412/2840] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 09:27:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 09:27:48.430901 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db25b6d9-fe53-44d3-a9fc-f3796c709140 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:27:48.441801 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db25b6d9-fe53-44d3-a9fc-f3796c709140 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:27:48.446698 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f159341f-a59b-4ee1-b453-dde349a3fc43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:27:48.447309 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f159341f-a59b-4ee1-b453-dde349a3fc43 admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:27:48.447662 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f159341f-a59b-4ee1-b453-dde349a3fc43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:27:48.447935 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f159341f-a59b-4ee1-b453-dde349a3fc43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:27:48.448578 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f159341f-a59b-4ee1-b453-dde349a3fc43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:27:48.450383 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f159341f-a59b-4ee1-b453-dde349a3fc43 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:27:48.465794 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1429/2841] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 09:27:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 09:27:48.465957 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1d8d46-ec94-48be-b823-70288f67a5b4 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:27:48.477167 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1d8d46-ec94-48be-b823-70288f67a5b4 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:27:48.481735 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07c29ed-4002-4523-9fce-aa9871b3ce62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:27:48.482422 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07c29ed-4002-4523-9fce-aa9871b3ce62 admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:27:48.482703 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07c29ed-4002-4523-9fce-aa9871b3ce62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:27:48.482975 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07c29ed-4002-4523-9fce-aa9871b3ce62 admin admin] PATH_INFO: `/v3/projects/a0cfc3bcd6654eb586aed0a9fe6c4e3e/users/b6124bc6c452457d8503512d3b1baea2/roles/44b64db484f24a46bf0b7ced46b519bf` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:27:48.503358 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07c29ed-4002-4523-9fce-aa9871b3ce62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0cfc3bcd6654eb586aed0a9fe6c4e3e, user_id=b6124bc6c452457d8503512d3b1baea2, role_id=44b64db484f24a46bf0b7ced46b519bf)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:27:48.506856 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07c29ed-4002-4523-9fce-aa9871b3ce62 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:27:48.584726 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1413/2842] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 09:27:48 2022] PUT /identity/v3/projects/a0cfc3bcd6654eb586aed0a9fe6c4e3e/users/b6124bc6c452457d8503512d3b1baea2/roles/44b64db484f24a46bf0b7ced46b519bf => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 09:27:48.594793 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bd4630-d214-4840-91bf-0521217de71e None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:27:48.595154 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bd4630-d214-4840-91bf-0521217de71e None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:27:48.595497 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bd4630-d214-4840-91bf-0521217de71e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:27:48.625315 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-92bd4630-d214-4840-91bf-0521217de71e None None] MFA Rules not processed for user `b6124bc6c452457d8503512d3b1baea2`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:27:48.626638 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-92bd4630-d214-4840-91bf-0521217de71e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:27:48.671121 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:27:48.692758 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1430/2843] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 09:27:48 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:27:48.697381 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07c29ed-4002-4523-9fce-aa9871b3ce62 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:27:48.763017 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07c29ed-4002-4523-9fce-aa9871b3ce62 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:27:48.767847 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc297d3-52b0-4feb-b2b4-63b79194b51a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:27:48.769123 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc297d3-52b0-4feb-b2b4-63b79194b51a admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:27:48.769123 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc297d3-52b0-4feb-b2b4-63b79194b51a admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:27:48.769123 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc297d3-52b0-4feb-b2b4-63b79194b51a admin admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:27:48.770230 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc297d3-52b0-4feb-b2b4-63b79194b51a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:27:48.772243 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc297d3-52b0-4feb-b2b4-63b79194b51a admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:27:48.888062 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1414/2844] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 09:27:48 2022] POST /identity/v3/projects => generated 367 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 09:27:48.892846 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bd4630-d214-4840-91bf-0521217de71e None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:27:48.905260 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bd4630-d214-4840-91bf-0521217de71e None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:27:49.013676 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce8f4f7-ea24-4db4-bc2f-427ae6d97426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:27:49.014298 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce8f4f7-ea24-4db4-bc2f-427ae6d97426 admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:27:49.014567 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce8f4f7-ea24-4db4-bc2f-427ae6d97426 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:27:49.014831 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce8f4f7-ea24-4db4-bc2f-427ae6d97426 admin admin] PATH_INFO: `/v3/users` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:27:49.015457 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce8f4f7-ea24-4db4-bc2f-427ae6d97426 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:27:49.018073 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce8f4f7-ea24-4db4-bc2f-427ae6d97426 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:27:49.109370 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1431/2845] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 09:27:48 2022] POST /identity/v3/users => generated 290 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 09:27:49.114018 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc297d3-52b0-4feb-b2b4-63b79194b51a admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:27:49.129402 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:27:49.129402 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc297d3-52b0-4feb-b2b4-63b79194b51a admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:27:49.135374 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e647a9b-4a9f-4ddb-a996-173c795a8500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:27:49.136118 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e647a9b-4a9f-4ddb-a996-173c795a8500 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:27:49.136464 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e647a9b-4a9f-4ddb-a996-173c795a8500 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:27:49.136806 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e647a9b-4a9f-4ddb-a996-173c795a8500 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:27:49.137547 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e647a9b-4a9f-4ddb-a996-173c795a8500 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:27:49.139392 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e647a9b-4a9f-4ddb-a996-173c795a8500 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:27:49.158197 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1415/2846] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 09:27:49 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 09:27:49.163327 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce8f4f7-ea24-4db4-bc2f-427ae6d97426 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:27:49.174174 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce8f4f7-ea24-4db4-bc2f-427ae6d97426 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:27:49.183522 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a56605-c282-4ab0-8de7-5738a4308114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:27:49.186095 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a56605-c282-4ab0-8de7-5738a4308114 admin admin] REQUEST_METHOD: `PUT` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:27:49.186095 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a56605-c282-4ab0-8de7-5738a4308114 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:27:49.186095 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a56605-c282-4ab0-8de7-5738a4308114 admin admin] PATH_INFO: `/v3/projects/1bc6780456e448f79002627fe5ae619f/users/80fa91e8a13f47fd80d3e53cd9da5c5d/roles/ec3751692d9c4ecf81b4a71c936c46be` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:27:49.223720 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a56605-c282-4ab0-8de7-5738a4308114 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bc6780456e448f79002627fe5ae619f, user_id=80fa91e8a13f47fd80d3e53cd9da5c5d, role_id=ec3751692d9c4ecf81b4a71c936c46be)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:27:49.227137 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a56605-c282-4ab0-8de7-5738a4308114 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:27:49.339841 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1432/2847] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 09:27:49 2022] PUT /identity/v3/projects/1bc6780456e448f79002627fe5ae619f/users/80fa91e8a13f47fd80d3e53cd9da5c5d/roles/ec3751692d9c4ecf81b4a71c936c46be => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:27:49.352093 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e647a9b-4a9f-4ddb-a996-173c795a8500 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:27:49.378343 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e647a9b-4a9f-4ddb-a996-173c795a8500 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:27:49.443518 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae63f320-1b6e-4d4f-bb97-be0f875e6cbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:27:49.444129 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae63f320-1b6e-4d4f-bb97-be0f875e6cbf admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:27:49.444407 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae63f320-1b6e-4d4f-bb97-be0f875e6cbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:27:49.444675 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae63f320-1b6e-4d4f-bb97-be0f875e6cbf admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:27:49.445483 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae63f320-1b6e-4d4f-bb97-be0f875e6cbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:27:49.447068 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae63f320-1b6e-4d4f-bb97-be0f875e6cbf admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:27:49.460551 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1416/2848] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 09:27:49 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 09:27:49.464637 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a56605-c282-4ab0-8de7-5738a4308114 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:27:49.475033 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a56605-c282-4ab0-8de7-5738a4308114 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:27:49.479804 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83824998-1e2f-492b-938e-0eb1a7d9859a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:27:49.480453 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83824998-1e2f-492b-938e-0eb1a7d9859a admin admin] REQUEST_METHOD: `PUT` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:27:49.480737 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83824998-1e2f-492b-938e-0eb1a7d9859a admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:27:49.481043 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83824998-1e2f-492b-938e-0eb1a7d9859a admin admin] PATH_INFO: `/v3/projects/1bc6780456e448f79002627fe5ae619f/users/80fa91e8a13f47fd80d3e53cd9da5c5d/roles/ec3751692d9c4ecf81b4a71c936c46be` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:27:49.483108 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83824998-1e2f-492b-938e-0eb1a7d9859a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bc6780456e448f79002627fe5ae619f, user_id=80fa91e8a13f47fd80d3e53cd9da5c5d, role_id=ec3751692d9c4ecf81b4a71c936c46be)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:27:49.485831 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83824998-1e2f-492b-938e-0eb1a7d9859a admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:27:49.497155 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1433/2849] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 09:27:49 2022] PUT /identity/v3/projects/1bc6780456e448f79002627fe5ae619f/users/80fa91e8a13f47fd80d3e53cd9da5c5d/roles/ec3751692d9c4ecf81b4a71c936c46be => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:27:49.508598 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5573cf-af26-4540-9550-c284c3a3d124 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:27:49.515334 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5573cf-af26-4540-9550-c284c3a3d124 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:27:49.515334 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5573cf-af26-4540-9550-c284c3a3d124 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:27:49.543894 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-ab5573cf-af26-4540-9550-c284c3a3d124 None None] MFA Rules not processed for user `80fa91e8a13f47fd80d3e53cd9da5c5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:27:49.545284 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-ab5573cf-af26-4540-9550-c284c3a3d124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:27:49.614842 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1417/2850] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 09:27:49 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:27:49.625272 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83824998-1e2f-492b-938e-0eb1a7d9859a admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:27:49.626942 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-83824998-1e2f-492b-938e-0eb1a7d9859a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:27:49.699322 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83824998-1e2f-492b-938e-0eb1a7d9859a admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:27:49.703002 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddc908b-1dcd-434a-aea8-405f1f321f9a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ba74e28ced4cbba857f54222dfbe41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:27:49.703613 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddc908b-1dcd-434a-aea8-405f1f321f9a service cinder] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:27:49.703898 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddc908b-1dcd-434a-aea8-405f1f321f9a service cinder] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:27:49.704165 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddc908b-1dcd-434a-aea8-405f1f321f9a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:27:49.713921 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddc908b-1dcd-434a-aea8-405f1f321f9a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:27:49.715362 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddc908b-1dcd-434a-aea8-405f1f321f9a service cinder] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:27:49.725846 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1434/2851] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:27:49 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:27:52.791287 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5573cf-af26-4540-9550-c284c3a3d124 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:27:52.854911 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5573cf-af26-4540-9550-c284c3a3d124 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:27:52.859300 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c3262c-f9c4-4860-965a-28cf356cc83f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:27:52.859915 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c3262c-f9c4-4860-965a-28cf356cc83f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:27:52.860190 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c3262c-f9c4-4860-965a-28cf356cc83f admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:27:52.860456 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c3262c-f9c4-4860-965a-28cf356cc83f admin admin] PATH_INFO: `/v3/users/b6124bc6c452457d8503512d3b1baea2` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:27:52.861576 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c3262c-f9c4-4860-965a-28cf356cc83f admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6124bc6c452457d8503512d3b1baea2)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:27:52.863633 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c3262c-f9c4-4860-965a-28cf356cc83f admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:27:52.879794 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:27:53.109206 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-30c3262c-f9c4-4860-965a-28cf356cc83f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6124bc6c452457d8503512d3b1baea2'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:27:53.123327 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-30c3262c-f9c4-4860-965a-28cf356cc83f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6124bc6c452457d8503512d3b1baea2'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:27:53.132330 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-30c3262c-f9c4-4860-965a-28cf356cc83f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6124bc6c452457d8503512d3b1baea2'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:27:53.133110 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-30c3262c-f9c4-4860-965a-28cf356cc83f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6124bc6c452457d8503512d3b1baea2'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:27:53.136547 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-30c3262c-f9c4-4860-965a-28cf356cc83f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6124bc6c452457d8503512d3b1baea2'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:27:53.225029 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1418/2852] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 09:27:52 2022] DELETE /identity/v3/users/b6124bc6c452457d8503512d3b1baea2 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:27:53.230772 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3710d02a-5f87-4bc1-a991-546c7081a349 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:27:53.231359 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3710d02a-5f87-4bc1-a991-546c7081a349 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:27:53.231923 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3710d02a-5f87-4bc1-a991-546c7081a349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:27:53.256551 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-3710d02a-5f87-4bc1-a991-546c7081a349 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:27:53.258296 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-3710d02a-5f87-4bc1-a991-546c7081a349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:27:53.299760 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:27:53.328130 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1435/2853] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 09:27:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 09:27:53.339078 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c3262c-f9c4-4860-965a-28cf356cc83f admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:27:53.340597 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-30c3262c-f9c4-4860-965a-28cf356cc83f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:27:53.408230 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c3262c-f9c4-4860-965a-28cf356cc83f admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:27:53.413124 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467c6ca0-6f9a-4c0f-b9da-d6579a68898d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:27:53.418032 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467c6ca0-6f9a-4c0f-b9da-d6579a68898d service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:27:53.418379 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467c6ca0-6f9a-4c0f-b9da-d6579a68898d service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:27:53.419791 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467c6ca0-6f9a-4c0f-b9da-d6579a68898d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:27:53.436388 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467c6ca0-6f9a-4c0f-b9da-d6579a68898d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:27:53.438557 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467c6ca0-6f9a-4c0f-b9da-d6579a68898d service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:27:53.453612 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1419/2854] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:27:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:27:53.589921 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3710d02a-5f87-4bc1-a991-546c7081a349 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:27:53.591465 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-3710d02a-5f87-4bc1-a991-546c7081a349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:27:53.674139 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3710d02a-5f87-4bc1-a991-546c7081a349 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:27:53.680017 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e03595-e20a-4467-bf0d-0b2b7e09d394 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2912307c56594125b157f1cfdf7c96a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:27:53.680612 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e03595-e20a-4467-bf0d-0b2b7e09d394 service placement] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:27:53.680891 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e03595-e20a-4467-bf0d-0b2b7e09d394 service placement] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:27:53.681194 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e03595-e20a-4467-bf0d-0b2b7e09d394 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:27:53.683342 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-47e03595-e20a-4467-bf0d-0b2b7e09d394 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:27:53.701123 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e03595-e20a-4467-bf0d-0b2b7e09d394 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:27:53.703034 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e03595-e20a-4467-bf0d-0b2b7e09d394 service placement] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:27:53.715643 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1436/2855] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:27:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:27:53.748043 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467c6ca0-6f9a-4c0f-b9da-d6579a68898d service neutron] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:27:53.749587 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-467c6ca0-6f9a-4c0f-b9da-d6579a68898d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:27:53.762603 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467c6ca0-6f9a-4c0f-b9da-d6579a68898d service neutron] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:27:53.767161 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f12f561-3c42-4896-b726-cff119fbb3ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:27:53.767788 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f12f561-3c42-4896-b726-cff119fbb3ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:27:53.768069 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f12f561-3c42-4896-b726-cff119fbb3ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:27:53.768363 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f12f561-3c42-4896-b726-cff119fbb3ae admin admin] PATH_INFO: `/v3/users/948d87bd5ba04663ac3fa8be3926d67a` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:27:53.769620 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f12f561-3c42-4896-b726-cff119fbb3ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=948d87bd5ba04663ac3fa8be3926d67a)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:27:53.771556 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f12f561-3c42-4896-b726-cff119fbb3ae admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:27:53.961718 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e03595-e20a-4467-bf0d-0b2b7e09d394 service placement] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:27:53.963213 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-47e03595-e20a-4467-bf0d-0b2b7e09d394 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:27:53.977477 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e03595-e20a-4467-bf0d-0b2b7e09d394 service placement] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:27:53.982774 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5ea528-d359-4241-a2f8-e2073ab49978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:27:53.983470 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5ea528-d359-4241-a2f8-e2073ab49978 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:27:53.984253 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5ea528-d359-4241-a2f8-e2073ab49978 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:27:53.984525 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5ea528-d359-4241-a2f8-e2073ab49978 admin admin] PATH_INFO: `/v3/projects/a0cfc3bcd6654eb586aed0a9fe6c4e3e` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:27:53.985713 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5ea528-d359-4241-a2f8-e2073ab49978 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0cfc3bcd6654eb586aed0a9fe6c4e3e)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:27:53.987884 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5ea528-d359-4241-a2f8-e2073ab49978 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:27:54.032179 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-5f12f561-3c42-4896-b726-cff119fbb3ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '948d87bd5ba04663ac3fa8be3926d67a'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:27:54.045868 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-5f12f561-3c42-4896-b726-cff119fbb3ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '948d87bd5ba04663ac3fa8be3926d67a'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:27:54.054085 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-5f12f561-3c42-4896-b726-cff119fbb3ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '948d87bd5ba04663ac3fa8be3926d67a'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:27:54.055078 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-5f12f561-3c42-4896-b726-cff119fbb3ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '948d87bd5ba04663ac3fa8be3926d67a'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:27:54.055696 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-5f12f561-3c42-4896-b726-cff119fbb3ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '948d87bd5ba04663ac3fa8be3926d67a'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:27:54.085257 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-ae5ea528-d359-4241-a2f8-e2073ab49978 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0cfc3bcd6654eb586aed0a9fe6c4e3e', 'request_id': 'req-ae5ea528-d359-4241-a2f8-e2073ab49978'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:27:54.090476 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-ae5ea528-d359-4241-a2f8-e2073ab49978 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0cfc3bcd6654eb586aed0a9fe6c4e3e', 'request_id': 'req-ae5ea528-d359-4241-a2f8-e2073ab49978'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:27:54.120603 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1420/2856] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 09:27:53 2022] DELETE /identity/v3/users/948d87bd5ba04663ac3fa8be3926d67a => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:27:54.153468 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-ae5ea528-d359-4241-a2f8-e2073ab49978 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0cfc3bcd6654eb586aed0a9fe6c4e3e', 'request_id': 'req-ae5ea528-d359-4241-a2f8-e2073ab49978'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:27:54.166628 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-ae5ea528-d359-4241-a2f8-e2073ab49978 admin admin] The token cache is being invalidate because project a0cfc3bcd6654eb586aed0a9fe6c4e3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 09:27:54.166918 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-ae5ea528-d359-4241-a2f8-e2073ab49978 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:27:54.167568 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-ae5ea528-d359-4241-a2f8-e2073ab49978 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:27:54.169208 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1437/2857] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 09:27:53 2022] DELETE /identity/v3/projects/a0cfc3bcd6654eb586aed0a9fe6c4e3e => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 09:27:54.173183 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f12f561-3c42-4896-b726-cff119fbb3ae admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:27:54.174612 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-5f12f561-3c42-4896-b726-cff119fbb3ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:27:54.187951 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:DELETE=19,keystone:SELECT=79,keystone:INSERT=2 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:27:54.253821 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f12f561-3c42-4896-b726-cff119fbb3ae admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:27:54.259215 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091ba6ed-3acb-40c8-8866-11bbe07c6ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:27:54.259998 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091ba6ed-3acb-40c8-8866-11bbe07c6ee1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:27:54.260304 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091ba6ed-3acb-40c8-8866-11bbe07c6ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:27:54.260575 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091ba6ed-3acb-40c8-8866-11bbe07c6ee1 admin admin] PATH_INFO: `/v3/users/80fa91e8a13f47fd80d3e53cd9da5c5d` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:27:54.265968 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091ba6ed-3acb-40c8-8866-11bbe07c6ee1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=80fa91e8a13f47fd80d3e53cd9da5c5d)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:27:54.265968 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091ba6ed-3acb-40c8-8866-11bbe07c6ee1 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:27:54.420321 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5ea528-d359-4241-a2f8-e2073ab49978 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:27:54.421756 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-ae5ea528-d359-4241-a2f8-e2073ab49978 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:27:54.477659 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:27:54.491003 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5ea528-d359-4241-a2f8-e2073ab49978 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:27:54.501264 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c985cb-c698-4e59-9a9b-07fa9b79242f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:27:54.501994 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c985cb-c698-4e59-9a9b-07fa9b79242f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:27:54.502278 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c985cb-c698-4e59-9a9b-07fa9b79242f admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:27:54.502543 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c985cb-c698-4e59-9a9b-07fa9b79242f admin admin] PATH_INFO: `/v3/projects/15117b97af5f40bbb1ba72a989ba5cab` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:27:54.503846 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c985cb-c698-4e59-9a9b-07fa9b79242f admin admin] RBAC: Authorizing `identity:delete_project(project_id=15117b97af5f40bbb1ba72a989ba5cab)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:27:54.506123 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c985cb-c698-4e59-9a9b-07fa9b79242f admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:27:54.509698 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-091ba6ed-3acb-40c8-8866-11bbe07c6ee1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80fa91e8a13f47fd80d3e53cd9da5c5d'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:27:54.523904 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-091ba6ed-3acb-40c8-8866-11bbe07c6ee1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80fa91e8a13f47fd80d3e53cd9da5c5d'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:27:54.531246 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-091ba6ed-3acb-40c8-8866-11bbe07c6ee1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80fa91e8a13f47fd80d3e53cd9da5c5d'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:27:54.531973 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-091ba6ed-3acb-40c8-8866-11bbe07c6ee1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80fa91e8a13f47fd80d3e53cd9da5c5d'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:27:54.532614 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-091ba6ed-3acb-40c8-8866-11bbe07c6ee1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80fa91e8a13f47fd80d3e53cd9da5c5d'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:27:54.613199 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1421/2858] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 09:27:54 2022] DELETE /identity/v3/users/80fa91e8a13f47fd80d3e53cd9da5c5d => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:27:54.630772 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-14c985cb-c698-4e59-9a9b-07fa9b79242f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15117b97af5f40bbb1ba72a989ba5cab', 'request_id': 'req-14c985cb-c698-4e59-9a9b-07fa9b79242f'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:27:54.637401 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-14c985cb-c698-4e59-9a9b-07fa9b79242f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15117b97af5f40bbb1ba72a989ba5cab', 'request_id': 'req-14c985cb-c698-4e59-9a9b-07fa9b79242f'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:27:54.708659 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-14c985cb-c698-4e59-9a9b-07fa9b79242f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15117b97af5f40bbb1ba72a989ba5cab', 'request_id': 'req-14c985cb-c698-4e59-9a9b-07fa9b79242f'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:27:54.721646 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-14c985cb-c698-4e59-9a9b-07fa9b79242f admin admin] The token cache is being invalidate because project 15117b97af5f40bbb1ba72a989ba5cab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 09:27:54.721936 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-14c985cb-c698-4e59-9a9b-07fa9b79242f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:27:54.723315 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-14c985cb-c698-4e59-9a9b-07fa9b79242f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:27:54.724838 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1438/2859] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 09:27:54 2022] DELETE /identity/v3/projects/15117b97af5f40bbb1ba72a989ba5cab => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 09:27:54.729072 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091ba6ed-3acb-40c8-8866-11bbe07c6ee1 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:27:54.730485 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-091ba6ed-3acb-40c8-8866-11bbe07c6ee1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:27:54.812262 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091ba6ed-3acb-40c8-8866-11bbe07c6ee1 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:27:54.864874 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:27:54.896766 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8f62b7-8294-4c5d-b6b2-f95cb2de3e8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:27:54.898078 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8f62b7-8294-4c5d-b6b2-f95cb2de3e8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:27:54.898359 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8f62b7-8294-4c5d-b6b2-f95cb2de3e8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:27:54.898636 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8f62b7-8294-4c5d-b6b2-f95cb2de3e8e admin admin] PATH_INFO: `/v3/users/d8367dfbd2d141d5acf9ae3fa7f6b6ea` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:27:54.899827 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8f62b7-8294-4c5d-b6b2-f95cb2de3e8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8367dfbd2d141d5acf9ae3fa7f6b6ea)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:27:54.901833 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8f62b7-8294-4c5d-b6b2-f95cb2de3e8e admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:27:55.060751 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c985cb-c698-4e59-9a9b-07fa9b79242f admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:27:55.062211 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-14c985cb-c698-4e59-9a9b-07fa9b79242f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:27:55.082814 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c985cb-c698-4e59-9a9b-07fa9b79242f admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:27:55.090388 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8be4f7-27b2-42e5-bbe8-75b849c50a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:27:55.091732 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8be4f7-27b2-42e5-bbe8-75b849c50a0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:27:55.092009 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8be4f7-27b2-42e5-bbe8-75b849c50a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:27:55.092347 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8be4f7-27b2-42e5-bbe8-75b849c50a0e admin admin] PATH_INFO: `/v3/projects/1bc6780456e448f79002627fe5ae619f` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:27:55.093623 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8be4f7-27b2-42e5-bbe8-75b849c50a0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=1bc6780456e448f79002627fe5ae619f)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:27:55.098308 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8be4f7-27b2-42e5-bbe8-75b849c50a0e admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:27:55.106642 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-fb8f62b7-8294-4c5d-b6b2-f95cb2de3e8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8367dfbd2d141d5acf9ae3fa7f6b6ea'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:27:55.120236 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-fb8f62b7-8294-4c5d-b6b2-f95cb2de3e8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8367dfbd2d141d5acf9ae3fa7f6b6ea'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:27:55.127265 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-fb8f62b7-8294-4c5d-b6b2-f95cb2de3e8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8367dfbd2d141d5acf9ae3fa7f6b6ea'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:27:55.127986 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-fb8f62b7-8294-4c5d-b6b2-f95cb2de3e8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8367dfbd2d141d5acf9ae3fa7f6b6ea'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:27:55.128585 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-fb8f62b7-8294-4c5d-b6b2-f95cb2de3e8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8367dfbd2d141d5acf9ae3fa7f6b6ea'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:27:55.196833 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-3b8be4f7-27b2-42e5-bbe8-75b849c50a0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1bc6780456e448f79002627fe5ae619f', 'request_id': 'req-3b8be4f7-27b2-42e5-bbe8-75b849c50a0e'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:27:55.203097 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-3b8be4f7-27b2-42e5-bbe8-75b849c50a0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1bc6780456e448f79002627fe5ae619f', 'request_id': 'req-3b8be4f7-27b2-42e5-bbe8-75b849c50a0e'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:27:55.208889 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1422/2860] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 09:27:54 2022] DELETE /identity/v3/users/d8367dfbd2d141d5acf9ae3fa7f6b6ea => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:27:55.292747 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-3b8be4f7-27b2-42e5-bbe8-75b849c50a0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1bc6780456e448f79002627fe5ae619f', 'request_id': 'req-3b8be4f7-27b2-42e5-bbe8-75b849c50a0e'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:27:55.306050 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-3b8be4f7-27b2-42e5-bbe8-75b849c50a0e admin admin] The token cache is being invalidate because project 1bc6780456e448f79002627fe5ae619f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 09:27:55.306590 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-3b8be4f7-27b2-42e5-bbe8-75b849c50a0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:27:55.307258 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-3b8be4f7-27b2-42e5-bbe8-75b849c50a0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:27:55.308832 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1439/2861] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 09:27:55 2022] DELETE /identity/v3/projects/1bc6780456e448f79002627fe5ae619f => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 09:27:55.313856 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c34287-59bc-4775-b5d9-e5673c85da2b None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:27:55.314310 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c34287-59bc-4775-b5d9-e5673c85da2b None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:27:55.314647 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c34287-59bc-4775-b5d9-e5673c85da2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:27:55.336849 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-e5c34287-59bc-4775-b5d9-e5673c85da2b None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:27:55.338112 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-e5c34287-59bc-4775-b5d9-e5673c85da2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:27:55.401707 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1423/2862] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 09:27:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:27:55.409748 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c552ba27-e58c-46df-aa34-14faa4bf5f6f None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:27:55.410088 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c552ba27-e58c-46df-aa34-14faa4bf5f6f None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:27:55.410540 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c552ba27-e58c-46df-aa34-14faa4bf5f6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:27:55.430309 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-c552ba27-e58c-46df-aa34-14faa4bf5f6f None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:27:55.431766 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-c552ba27-e58c-46df-aa34-14faa4bf5f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:27:55.440125 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1440/2863] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 09:27:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:27:55.445227 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c34287-59bc-4775-b5d9-e5673c85da2b None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:27:55.449425 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c552ba27-e58c-46df-aa34-14faa4bf5f6f None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:27:55.450991 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-c552ba27-e58c-46df-aa34-14faa4bf5f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:27:55.455183 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c34287-59bc-4775-b5d9-e5673c85da2b None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:27:55.460908 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716f3c81-6f1c-4a5e-bff2-9c679537d9f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:27:55.461413 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716f3c81-6f1c-4a5e-bff2-9c679537d9f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:27:55.461691 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716f3c81-6f1c-4a5e-bff2-9c679537d9f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:27:55.461960 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716f3c81-6f1c-4a5e-bff2-9c679537d9f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:27:55.462631 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716f3c81-6f1c-4a5e-bff2-9c679537d9f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:27:55.462924 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716f3c81-6f1c-4a5e-bff2-9c679537d9f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:27:55.464702 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716f3c81-6f1c-4a5e-bff2-9c679537d9f5 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:27:55.465196 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c552ba27-e58c-46df-aa34-14faa4bf5f6f None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:27:55.471049 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ebe50c-c647-45e5-8a8c-a4ab6c74a823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:27:55.471691 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ebe50c-c647-45e5-8a8c-a4ab6c74a823 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:27:55.471972 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ebe50c-c647-45e5-8a8c-a4ab6c74a823 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:27:55.472266 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ebe50c-c647-45e5-8a8c-a4ab6c74a823 admin admin] PATH_INFO: `/v3/projects/8593f19e8bbd4afaa55a6c859b184e58` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:27:55.473634 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ebe50c-c647-45e5-8a8c-a4ab6c74a823 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8593f19e8bbd4afaa55a6c859b184e58)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:27:55.475589 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ebe50c-c647-45e5-8a8c-a4ab6c74a823 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:27:55.478189 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1424/2864] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Aug 10 09:27:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 09:27:55.483573 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716f3c81-6f1c-4a5e-bff2-9c679537d9f5 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:27:55.494810 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716f3c81-6f1c-4a5e-bff2-9c679537d9f5 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:27:55.499477 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc902b8-d511-43ba-8466-7813f6d1e895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:27:55.500059 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc902b8-d511-43ba-8466-7813f6d1e895 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:27:55.500334 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc902b8-d511-43ba-8466-7813f6d1e895 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:27:55.500598 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc902b8-d511-43ba-8466-7813f6d1e895 admin admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:27:55.501250 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc902b8-d511-43ba-8466-7813f6d1e895 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:27:55.508996 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc902b8-d511-43ba-8466-7813f6d1e895 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:27:55.593837 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-13ebe50c-c647-45e5-8a8c-a4ab6c74a823 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8593f19e8bbd4afaa55a6c859b184e58', 'request_id': 'req-13ebe50c-c647-45e5-8a8c-a4ab6c74a823'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:27:55.597766 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:27:55.599811 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-13ebe50c-c647-45e5-8a8c-a4ab6c74a823 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8593f19e8bbd4afaa55a6c859b184e58', 'request_id': 'req-13ebe50c-c647-45e5-8a8c-a4ab6c74a823'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:27:55.612415 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1425/2865] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 09:27:55 2022] POST /identity/v3/projects => generated 401 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 09:27:55.618034 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc902b8-d511-43ba-8466-7813f6d1e895 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:27:55.639929 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:27:55.681839 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc902b8-d511-43ba-8466-7813f6d1e895 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:27:55.686460 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121a8cc8-5b38-4581-a8aa-5007bc07c617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:27:55.687055 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121a8cc8-5b38-4581-a8aa-5007bc07c617 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:27:55.687331 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121a8cc8-5b38-4581-a8aa-5007bc07c617 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:27:55.687599 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121a8cc8-5b38-4581-a8aa-5007bc07c617 admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:27:55.688728 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121a8cc8-5b38-4581-a8aa-5007bc07c617 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:27:55.690153 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121a8cc8-5b38-4581-a8aa-5007bc07c617 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:27:55.708140 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-13ebe50c-c647-45e5-8a8c-a4ab6c74a823 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8593f19e8bbd4afaa55a6c859b184e58', 'request_id': 'req-13ebe50c-c647-45e5-8a8c-a4ab6c74a823'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:27:55.719580 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-13ebe50c-c647-45e5-8a8c-a4ab6c74a823 admin admin] The token cache is being invalidate because project 8593f19e8bbd4afaa55a6c859b184e58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 09:27:55.719912 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-13ebe50c-c647-45e5-8a8c-a4ab6c74a823 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:27:55.720620 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-13ebe50c-c647-45e5-8a8c-a4ab6c74a823 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:27:55.722144 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1441/2866] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 09:27:55 2022] DELETE /identity/v3/projects/8593f19e8bbd4afaa55a6c859b184e58 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 09:27:55.789541 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1426/2867] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 09:27:55 2022] POST /identity/v3/users => generated 301 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 09:27:55.793718 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ebe50c-c647-45e5-8a8c-a4ab6c74a823 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:27:55.795210 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-13ebe50c-c647-45e5-8a8c-a4ab6c74a823 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:27:55.868111 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ebe50c-c647-45e5-8a8c-a4ab6c74a823 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:27:55.873079 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b838df-8f86-4622-9d78-2b44f60506ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:27:55.873749 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b838df-8f86-4622-9d78-2b44f60506ea admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:27:55.874031 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b838df-8f86-4622-9d78-2b44f60506ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:27:55.874307 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b838df-8f86-4622-9d78-2b44f60506ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:27:55.874965 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b838df-8f86-4622-9d78-2b44f60506ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:27:55.876626 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b838df-8f86-4622-9d78-2b44f60506ea admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:27:55.887615 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1442/2868] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 09:27:55 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 09:27:55.892160 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121a8cc8-5b38-4581-a8aa-5007bc07c617 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:27:55.903025 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121a8cc8-5b38-4581-a8aa-5007bc07c617 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:27:55.907878 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fdb991-214c-4844-8d7a-f32b69aa1704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:27:55.909476 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fdb991-214c-4844-8d7a-f32b69aa1704 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:27:55.909476 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fdb991-214c-4844-8d7a-f32b69aa1704 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:27:55.909476 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fdb991-214c-4844-8d7a-f32b69aa1704 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:27:55.909730 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fdb991-214c-4844-8d7a-f32b69aa1704 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:27:55.912185 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fdb991-214c-4844-8d7a-f32b69aa1704 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:27:55.928762 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1427/2869] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 09:27:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 09:27:55.930041 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b838df-8f86-4622-9d78-2b44f60506ea admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:27:55.942798 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b838df-8f86-4622-9d78-2b44f60506ea admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:27:55.949895 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dea86d-e9e2-46e1-bf73-01914bfd8ecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:27:55.950582 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dea86d-e9e2-46e1-bf73-01914bfd8ecf admin admin] REQUEST_METHOD: `PUT` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:27:55.950868 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dea86d-e9e2-46e1-bf73-01914bfd8ecf admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:27:55.951145 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dea86d-e9e2-46e1-bf73-01914bfd8ecf admin admin] PATH_INFO: `/v3/projects/69b0f2425473495f9358bfabe800f298/users/57e98392dae946a7b2b70b4102750546/roles/44b64db484f24a46bf0b7ced46b519bf` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:27:55.968620 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dea86d-e9e2-46e1-bf73-01914bfd8ecf admin admin] RBAC: Authorizing `identity:create_grant(project_id=69b0f2425473495f9358bfabe800f298, user_id=57e98392dae946a7b2b70b4102750546, role_id=44b64db484f24a46bf0b7ced46b519bf)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:27:55.971773 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dea86d-e9e2-46e1-bf73-01914bfd8ecf admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:27:56.052084 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1443/2870] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 09:27:55 2022] PUT /identity/v3/projects/69b0f2425473495f9358bfabe800f298/users/57e98392dae946a7b2b70b4102750546/roles/44b64db484f24a46bf0b7ced46b519bf => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:27:56.063137 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487d2e2c-6c5f-4584-a17f-116320de5afa None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:27:56.063424 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487d2e2c-6c5f-4584-a17f-116320de5afa None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:27:56.063697 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487d2e2c-6c5f-4584-a17f-116320de5afa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:27:56.092656 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-487d2e2c-6c5f-4584-a17f-116320de5afa None None] MFA Rules not processed for user `57e98392dae946a7b2b70b4102750546`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:27:56.093899 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-487d2e2c-6c5f-4584-a17f-116320de5afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:27:56.160679 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1428/2871] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 09:27:56 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:27:56.165471 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dea86d-e9e2-46e1-bf73-01914bfd8ecf admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:27:56.232798 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dea86d-e9e2-46e1-bf73-01914bfd8ecf admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:27:56.237559 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910e8cf1-aed4-48a7-af23-7f51b9e35702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:27:56.238276 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910e8cf1-aed4-48a7-af23-7f51b9e35702 admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:27:56.238557 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910e8cf1-aed4-48a7-af23-7f51b9e35702 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:27:56.238885 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910e8cf1-aed4-48a7-af23-7f51b9e35702 admin admin] PATH_INFO: `/v3/projects` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:27:56.239411 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910e8cf1-aed4-48a7-af23-7f51b9e35702 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:27:56.241405 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910e8cf1-aed4-48a7-af23-7f51b9e35702 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:27:56.379442 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1444/2872] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 09:27:56 2022] POST /identity/v3/projects => generated 401 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 09:27:56.383209 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487d2e2c-6c5f-4584-a17f-116320de5afa None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:27:56.393976 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487d2e2c-6c5f-4584-a17f-116320de5afa None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:27:56.450479 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7afea1-6f82-4b6d-956f-3b9e76824223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:27:56.451080 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7afea1-6f82-4b6d-956f-3b9e76824223 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:27:56.451355 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7afea1-6f82-4b6d-956f-3b9e76824223 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:27:56.451622 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7afea1-6f82-4b6d-956f-3b9e76824223 admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:27:56.452245 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7afea1-6f82-4b6d-956f-3b9e76824223 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:27:56.454174 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7afea1-6f82-4b6d-956f-3b9e76824223 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:27:56.530374 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1429/2873] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 09:27:56 2022] POST /identity/v3/users => generated 307 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 09:27:56.534633 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910e8cf1-aed4-48a7-af23-7f51b9e35702 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:27:56.545738 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910e8cf1-aed4-48a7-af23-7f51b9e35702 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:27:56.551216 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8faa166-6ef8-4e75-9545-2aa5e55576f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:27:56.551970 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8faa166-6ef8-4e75-9545-2aa5e55576f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:27:56.552252 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8faa166-6ef8-4e75-9545-2aa5e55576f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:27:56.552524 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8faa166-6ef8-4e75-9545-2aa5e55576f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:27:56.553264 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8faa166-6ef8-4e75-9545-2aa5e55576f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:27:56.554973 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8faa166-6ef8-4e75-9545-2aa5e55576f4 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:27:56.567086 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1445/2874] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 09:27:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 09:27:56.571531 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7afea1-6f82-4b6d-956f-3b9e76824223 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:27:56.581749 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7afea1-6f82-4b6d-956f-3b9e76824223 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:27:56.586628 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67834680-25d7-4988-a783-cf6f15a851dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:27:56.587267 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67834680-25d7-4988-a783-cf6f15a851dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:27:56.587549 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67834680-25d7-4988-a783-cf6f15a851dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:27:56.587820 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67834680-25d7-4988-a783-cf6f15a851dd admin admin] PATH_INFO: `/v3/projects/251e401456154b9a8b2f6f1d6b954696/users/ce611520eaff4a3e82efdb8e1bd1b20e/roles/ec3751692d9c4ecf81b4a71c936c46be` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:27:56.603235 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67834680-25d7-4988-a783-cf6f15a851dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=251e401456154b9a8b2f6f1d6b954696, user_id=ce611520eaff4a3e82efdb8e1bd1b20e, role_id=ec3751692d9c4ecf81b4a71c936c46be)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:27:56.606066 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67834680-25d7-4988-a783-cf6f15a851dd admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:27:56.608241 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:27:56.674116 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1430/2875] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 09:27:56 2022] PUT /identity/v3/projects/251e401456154b9a8b2f6f1d6b954696/users/ce611520eaff4a3e82efdb8e1bd1b20e/roles/ec3751692d9c4ecf81b4a71c936c46be => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:27:56.678812 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8faa166-6ef8-4e75-9545-2aa5e55576f4 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:27:56.689291 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8faa166-6ef8-4e75-9545-2aa5e55576f4 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:27:56.714549 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:27:56.744811 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7628f31-b5de-466d-8c50-959b89e2ea47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:27:56.745444 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7628f31-b5de-466d-8c50-959b89e2ea47 admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:27:56.745722 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7628f31-b5de-466d-8c50-959b89e2ea47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:27:56.745993 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7628f31-b5de-466d-8c50-959b89e2ea47 admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:27:56.746626 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7628f31-b5de-466d-8c50-959b89e2ea47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:27:56.748257 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7628f31-b5de-466d-8c50-959b89e2ea47 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:27:56.758020 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1446/2876] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 09:27:56 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 09:27:56.762206 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67834680-25d7-4988-a783-cf6f15a851dd admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:27:56.772717 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67834680-25d7-4988-a783-cf6f15a851dd admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:27:56.777716 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc35616-8692-42ba-a891-ad1031cf1d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:27:56.778365 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc35616-8692-42ba-a891-ad1031cf1d03 admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:27:56.778646 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc35616-8692-42ba-a891-ad1031cf1d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:27:56.778917 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc35616-8692-42ba-a891-ad1031cf1d03 admin admin] PATH_INFO: `/v3/projects/251e401456154b9a8b2f6f1d6b954696/users/ce611520eaff4a3e82efdb8e1bd1b20e/roles/ec3751692d9c4ecf81b4a71c936c46be` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:27:56.780882 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc35616-8692-42ba-a891-ad1031cf1d03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=251e401456154b9a8b2f6f1d6b954696, user_id=ce611520eaff4a3e82efdb8e1bd1b20e, role_id=ec3751692d9c4ecf81b4a71c936c46be)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:27:56.783557 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc35616-8692-42ba-a891-ad1031cf1d03 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:27:56.794296 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1431/2877] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 09:27:56 2022] PUT /identity/v3/projects/251e401456154b9a8b2f6f1d6b954696/users/ce611520eaff4a3e82efdb8e1bd1b20e/roles/ec3751692d9c4ecf81b4a71c936c46be => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:27:56.809746 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1898d370-c842-4562-93f8-f537ace51021 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:27:56.810034 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1898d370-c842-4562-93f8-f537ace51021 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:27:56.810313 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1898d370-c842-4562-93f8-f537ace51021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:27:56.837823 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-1898d370-c842-4562-93f8-f537ace51021 None None] MFA Rules not processed for user `ce611520eaff4a3e82efdb8e1bd1b20e`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:27:56.839096 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-1898d370-c842-4562-93f8-f537ace51021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:27:56.912662 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1447/2878] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 09:27:56 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:27:56.924278 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc35616-8692-42ba-a891-ad1031cf1d03 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:27:56.925941 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-dcc35616-8692-42ba-a891-ad1031cf1d03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:27:56.988838 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc35616-8692-42ba-a891-ad1031cf1d03 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:27:56.992552 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe32140-fc0f-47a2-a0b8-7ace4d3a440d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ba74e28ced4cbba857f54222dfbe41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:27:56.993190 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe32140-fc0f-47a2-a0b8-7ace4d3a440d service cinder] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:27:56.993471 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe32140-fc0f-47a2-a0b8-7ace4d3a440d service cinder] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:27:56.993743 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe32140-fc0f-47a2-a0b8-7ace4d3a440d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:27:57.002558 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe32140-fc0f-47a2-a0b8-7ace4d3a440d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:27:57.003978 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe32140-fc0f-47a2-a0b8-7ace4d3a440d service cinder] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:27:57.013585 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1432/2879] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:27:56 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:27:57.251021 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1898d370-c842-4562-93f8-f537ace51021 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:27:57.261368 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1898d370-c842-4562-93f8-f537ace51021 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:27:57.264876 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10515cf2-ade6-4af3-9f8f-e04de9bbb53a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ba74e28ced4cbba857f54222dfbe41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:27:57.265493 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10515cf2-ade6-4af3-9f8f-e04de9bbb53a service cinder] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:27:57.265768 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10515cf2-ade6-4af3-9f8f-e04de9bbb53a service cinder] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:27:57.266035 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10515cf2-ade6-4af3-9f8f-e04de9bbb53a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:27:57.326201 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10515cf2-ade6-4af3-9f8f-e04de9bbb53a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:27:57.327683 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10515cf2-ade6-4af3-9f8f-e04de9bbb53a service cinder] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:27:57.337612 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1448/2880] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:27:57 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:28:06.478797 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe32140-fc0f-47a2-a0b8-7ace4d3a440d service cinder] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:28:06.540625 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe32140-fc0f-47a2-a0b8-7ace4d3a440d service cinder] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:28:06.545154 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653bce51-0893-4cd3-9d5d-c25184e2fdd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:28:06.545775 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653bce51-0893-4cd3-9d5d-c25184e2fdd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:28:06.546050 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653bce51-0893-4cd3-9d5d-c25184e2fdd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:28:06.546317 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653bce51-0893-4cd3-9d5d-c25184e2fdd6 admin admin] PATH_INFO: `/v3/users/57e98392dae946a7b2b70b4102750546` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:28:06.547427 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653bce51-0893-4cd3-9d5d-c25184e2fdd6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=57e98392dae946a7b2b70b4102750546)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:28:06.549387 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653bce51-0893-4cd3-9d5d-c25184e2fdd6 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:28:06.733761 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-653bce51-0893-4cd3-9d5d-c25184e2fdd6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57e98392dae946a7b2b70b4102750546'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:28:06.746204 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-653bce51-0893-4cd3-9d5d-c25184e2fdd6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57e98392dae946a7b2b70b4102750546'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:28:06.752937 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-653bce51-0893-4cd3-9d5d-c25184e2fdd6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57e98392dae946a7b2b70b4102750546'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:28:06.753714 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-653bce51-0893-4cd3-9d5d-c25184e2fdd6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57e98392dae946a7b2b70b4102750546'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:28:06.754284 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-653bce51-0893-4cd3-9d5d-c25184e2fdd6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57e98392dae946a7b2b70b4102750546'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:28:06.823723 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1433/2881] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 09:28:06 2022] DELETE /identity/v3/users/57e98392dae946a7b2b70b4102750546 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:28:06.828352 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3145865-8005-47a0-802d-7ebca48a98b7 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:28:06.828808 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3145865-8005-47a0-802d-7ebca48a98b7 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:28:06.828968 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3145865-8005-47a0-802d-7ebca48a98b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:28:06.849401 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-a3145865-8005-47a0-802d-7ebca48a98b7 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:28:06.850578 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-a3145865-8005-47a0-802d-7ebca48a98b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:28:06.892322 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=100 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:28:06.907766 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1449/2882] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 09:28:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:28:06.918894 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653bce51-0893-4cd3-9d5d-c25184e2fdd6 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:28:06.920333 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-653bce51-0893-4cd3-9d5d-c25184e2fdd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:28:06.925802 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:INSERT=3,keystone:SELECT=87,keystone:DELETE=10 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:28:06.981184 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653bce51-0893-4cd3-9d5d-c25184e2fdd6 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:28:06.985392 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52565dc6-ee2a-4abb-a3cb-c593ca0cc5d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:28:06.985933 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52565dc6-ee2a-4abb-a3cb-c593ca0cc5d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:28:06.986212 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52565dc6-ee2a-4abb-a3cb-c593ca0cc5d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:28:06.986481 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52565dc6-ee2a-4abb-a3cb-c593ca0cc5d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:28:06.995518 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52565dc6-ee2a-4abb-a3cb-c593ca0cc5d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:28:06.996962 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52565dc6-ee2a-4abb-a3cb-c593ca0cc5d2 service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:28:07.006271 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1434/2883] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:28:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:28:07.384329 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3145865-8005-47a0-802d-7ebca48a98b7 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:28:07.385835 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-a3145865-8005-47a0-802d-7ebca48a98b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:28:07.399627 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3145865-8005-47a0-802d-7ebca48a98b7 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:28:07.404798 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d880a9-bed7-483e-85bc-2e1720950629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:28:07.405108 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d880a9-bed7-483e-85bc-2e1720950629 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:28:07.405525 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d880a9-bed7-483e-85bc-2e1720950629 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:28:07.407204 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d880a9-bed7-483e-85bc-2e1720950629 admin admin] PATH_INFO: `/v3/projects/69b0f2425473495f9358bfabe800f298` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:28:07.407984 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d880a9-bed7-483e-85bc-2e1720950629 admin admin] RBAC: Authorizing `identity:delete_project(project_id=69b0f2425473495f9358bfabe800f298)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:28:07.410141 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d880a9-bed7-483e-85bc-2e1720950629 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:28:07.525546 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-68d880a9-bed7-483e-85bc-2e1720950629 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69b0f2425473495f9358bfabe800f298', 'request_id': 'req-68d880a9-bed7-483e-85bc-2e1720950629'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:28:07.530769 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-68d880a9-bed7-483e-85bc-2e1720950629 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69b0f2425473495f9358bfabe800f298', 'request_id': 'req-68d880a9-bed7-483e-85bc-2e1720950629'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:28:07.587836 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-68d880a9-bed7-483e-85bc-2e1720950629 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69b0f2425473495f9358bfabe800f298', 'request_id': 'req-68d880a9-bed7-483e-85bc-2e1720950629'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:28:07.598992 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-68d880a9-bed7-483e-85bc-2e1720950629 admin admin] The token cache is being invalidate because project 69b0f2425473495f9358bfabe800f298 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 09:28:07.599281 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-68d880a9-bed7-483e-85bc-2e1720950629 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:28:07.600456 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-68d880a9-bed7-483e-85bc-2e1720950629 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:28:07.601999 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1450/2884] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 09:28:07 2022] DELETE /identity/v3/projects/69b0f2425473495f9358bfabe800f298 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:28:07.606025 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52565dc6-ee2a-4abb-a3cb-c593ca0cc5d2 service neutron] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:28:07.615999 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52565dc6-ee2a-4abb-a3cb-c593ca0cc5d2 service neutron] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:28:07.672691 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074d1ab3-c493-4795-a834-90fa94a67b9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:28:07.673360 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074d1ab3-c493-4795-a834-90fa94a67b9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:28:07.673636 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074d1ab3-c493-4795-a834-90fa94a67b9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:28:07.673905 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074d1ab3-c493-4795-a834-90fa94a67b9b admin admin] PATH_INFO: `/v3/users/ce611520eaff4a3e82efdb8e1bd1b20e` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:28:07.675100 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074d1ab3-c493-4795-a834-90fa94a67b9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce611520eaff4a3e82efdb8e1bd1b20e)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:28:07.677087 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074d1ab3-c493-4795-a834-90fa94a67b9b admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:28:07.842659 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-074d1ab3-c493-4795-a834-90fa94a67b9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce611520eaff4a3e82efdb8e1bd1b20e'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:28:07.855153 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-074d1ab3-c493-4795-a834-90fa94a67b9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce611520eaff4a3e82efdb8e1bd1b20e'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:28:07.862192 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-074d1ab3-c493-4795-a834-90fa94a67b9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce611520eaff4a3e82efdb8e1bd1b20e'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:28:07.862958 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-074d1ab3-c493-4795-a834-90fa94a67b9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce611520eaff4a3e82efdb8e1bd1b20e'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:28:07.863552 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-074d1ab3-c493-4795-a834-90fa94a67b9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce611520eaff4a3e82efdb8e1bd1b20e'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:28:07.919270 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1435/2885] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 09:28:07 2022] DELETE /identity/v3/users/ce611520eaff4a3e82efdb8e1bd1b20e => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:28:08.332692 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d880a9-bed7-483e-85bc-2e1720950629 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:28:08.335428 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-68d880a9-bed7-483e-85bc-2e1720950629 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:28:08.352411 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d880a9-bed7-483e-85bc-2e1720950629 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:28:08.423676 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d62c2b-1c65-482d-b77e-b34006765d9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:28:08.424305 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d62c2b-1c65-482d-b77e-b34006765d9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:28:08.424581 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d62c2b-1c65-482d-b77e-b34006765d9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:28:08.424848 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d62c2b-1c65-482d-b77e-b34006765d9b admin admin] PATH_INFO: `/v3/projects/251e401456154b9a8b2f6f1d6b954696` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:28:08.426015 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d62c2b-1c65-482d-b77e-b34006765d9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=251e401456154b9a8b2f6f1d6b954696)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:28:08.439891 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d62c2b-1c65-482d-b77e-b34006765d9b admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:28:08.623931 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-c2d62c2b-1c65-482d-b77e-b34006765d9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '251e401456154b9a8b2f6f1d6b954696', 'request_id': 'req-c2d62c2b-1c65-482d-b77e-b34006765d9b'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:28:08.629188 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-c2d62c2b-1c65-482d-b77e-b34006765d9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '251e401456154b9a8b2f6f1d6b954696', 'request_id': 'req-c2d62c2b-1c65-482d-b77e-b34006765d9b'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:28:08.696172 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-c2d62c2b-1c65-482d-b77e-b34006765d9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '251e401456154b9a8b2f6f1d6b954696', 'request_id': 'req-c2d62c2b-1c65-482d-b77e-b34006765d9b'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:28:08.707522 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-c2d62c2b-1c65-482d-b77e-b34006765d9b admin admin] The token cache is being invalidate because project 251e401456154b9a8b2f6f1d6b954696 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 09:28:08.707826 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-c2d62c2b-1c65-482d-b77e-b34006765d9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:28:08.708497 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-c2d62c2b-1c65-482d-b77e-b34006765d9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:28:08.710780 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1451/2886] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 09:28:08 2022] DELETE /identity/v3/projects/251e401456154b9a8b2f6f1d6b954696 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 09:28:08.716471 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1eb0e4-9c18-4fbe-9d45-ae251c9a1467 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:28:08.721714 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1eb0e4-9c18-4fbe-9d45-ae251c9a1467 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:28:08.721714 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1eb0e4-9c18-4fbe-9d45-ae251c9a1467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:28:08.736222 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-8a1eb0e4-9c18-4fbe-9d45-ae251c9a1467 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:28:08.737448 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-8a1eb0e4-9c18-4fbe-9d45-ae251c9a1467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:28:08.755386 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:28:08.797668 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1436/2887] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 09:28:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:28:08.806214 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fba5a20-0a5b-47f4-ba6c-ab9f1fc81e8c None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:28:08.806597 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fba5a20-0a5b-47f4-ba6c-ab9f1fc81e8c None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:28:08.807242 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fba5a20-0a5b-47f4-ba6c-ab9f1fc81e8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:28:08.815004 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:28:08.827700 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-2fba5a20-0a5b-47f4-ba6c-ab9f1fc81e8c None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:28:08.828902 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-2fba5a20-0a5b-47f4-ba6c-ab9f1fc81e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:28:08.837738 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1452/2888] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 09:28:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:28:08.841906 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1eb0e4-9c18-4fbe-9d45-ae251c9a1467 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:28:08.852066 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1eb0e4-9c18-4fbe-9d45-ae251c9a1467 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:28:08.857042 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209d687a-9eb1-45ae-9210-5e21b609550b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:28:08.857640 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209d687a-9eb1-45ae-9210-5e21b609550b admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:28:08.857947 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209d687a-9eb1-45ae-9210-5e21b609550b admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:28:08.858222 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209d687a-9eb1-45ae-9210-5e21b609550b admin admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:28:08.858864 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209d687a-9eb1-45ae-9210-5e21b609550b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:28:08.859166 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209d687a-9eb1-45ae-9210-5e21b609550b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:28:08.860793 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209d687a-9eb1-45ae-9210-5e21b609550b admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:28:08.872442 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1437/2889] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Aug 10 09:28:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:28:08.877693 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fba5a20-0a5b-47f4-ba6c-ab9f1fc81e8c None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:28:08.886243 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fba5a20-0a5b-47f4-ba6c-ab9f1fc81e8c None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:28:08.890948 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5551179-a155-464c-b9fa-02edcfb2a2fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:28:08.891537 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5551179-a155-464c-b9fa-02edcfb2a2fd admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:28:08.891814 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5551179-a155-464c-b9fa-02edcfb2a2fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:28:08.892083 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5551179-a155-464c-b9fa-02edcfb2a2fd admin admin] PATH_INFO: `/v3/projects` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:28:08.892704 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5551179-a155-464c-b9fa-02edcfb2a2fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:28:08.894695 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5551179-a155-464c-b9fa-02edcfb2a2fd admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:28:08.992216 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1453/2890] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 09:28:08 2022] POST /identity/v3/projects => generated 385 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 09:28:08.997209 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209d687a-9eb1-45ae-9210-5e21b609550b admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:28:09.008053 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209d687a-9eb1-45ae-9210-5e21b609550b admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:28:09.065563 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fe46d3-636d-4308-b814-1e0e866e55ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:28:09.066195 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fe46d3-636d-4308-b814-1e0e866e55ed admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:28:09.066478 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fe46d3-636d-4308-b814-1e0e866e55ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:28:09.066751 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fe46d3-636d-4308-b814-1e0e866e55ed admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:28:09.067381 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fe46d3-636d-4308-b814-1e0e866e55ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:28:09.069948 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fe46d3-636d-4308-b814-1e0e866e55ed admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:28:09.177565 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1438/2891] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 09:28:08 2022] POST /identity/v3/users => generated 293 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 09:28:09.182487 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5551179-a155-464c-b9fa-02edcfb2a2fd admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:28:09.192680 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5551179-a155-464c-b9fa-02edcfb2a2fd admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:28:09.199513 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe23156-9842-4dcf-8320-cd9fe453510b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:28:09.206078 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe23156-9842-4dcf-8320-cd9fe453510b admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:28:09.206078 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe23156-9842-4dcf-8320-cd9fe453510b admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:28:09.206078 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe23156-9842-4dcf-8320-cd9fe453510b admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:28:09.206078 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe23156-9842-4dcf-8320-cd9fe453510b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:28:09.206078 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe23156-9842-4dcf-8320-cd9fe453510b admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:28:09.213113 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1454/2892] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 09:28:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 09:28:09.217202 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fe46d3-636d-4308-b814-1e0e866e55ed admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:28:09.227608 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fe46d3-636d-4308-b814-1e0e866e55ed admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:28:09.232444 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918c9a8a-45a2-4330-9919-d1ccd9cf6c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:28:09.233090 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918c9a8a-45a2-4330-9919-d1ccd9cf6c57 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:28:09.234837 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918c9a8a-45a2-4330-9919-d1ccd9cf6c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:28:09.234837 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918c9a8a-45a2-4330-9919-d1ccd9cf6c57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:28:09.234837 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918c9a8a-45a2-4330-9919-d1ccd9cf6c57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:28:09.236056 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918c9a8a-45a2-4330-9919-d1ccd9cf6c57 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:28:09.247362 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1439/2893] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 09:28:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 09:28:09.251369 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe23156-9842-4dcf-8320-cd9fe453510b admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:28:09.262362 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe23156-9842-4dcf-8320-cd9fe453510b admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:28:09.266985 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759abae3-c34c-4a21-8cf5-d8b3c57ef37c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:28:09.267622 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759abae3-c34c-4a21-8cf5-d8b3c57ef37c admin admin] REQUEST_METHOD: `PUT` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:28:09.267902 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759abae3-c34c-4a21-8cf5-d8b3c57ef37c admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:28:09.268174 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759abae3-c34c-4a21-8cf5-d8b3c57ef37c admin admin] PATH_INFO: `/v3/projects/cd7626373c8440acbed250fbaf2b5134/users/f9ba1e56fd13424687ffd2e357835a76/roles/44b64db484f24a46bf0b7ced46b519bf` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:28:09.282357 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759abae3-c34c-4a21-8cf5-d8b3c57ef37c admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd7626373c8440acbed250fbaf2b5134, user_id=f9ba1e56fd13424687ffd2e357835a76, role_id=44b64db484f24a46bf0b7ced46b519bf)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:28:09.285042 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759abae3-c34c-4a21-8cf5-d8b3c57ef37c admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:28:09.359515 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1455/2894] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 09:28:09 2022] PUT /identity/v3/projects/cd7626373c8440acbed250fbaf2b5134/users/f9ba1e56fd13424687ffd2e357835a76/roles/44b64db484f24a46bf0b7ced46b519bf => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:28:09.371902 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be4e42b-08e5-4e8a-8d33-476ab7cf18a3 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:28:09.372188 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be4e42b-08e5-4e8a-8d33-476ab7cf18a3 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:28:09.372466 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be4e42b-08e5-4e8a-8d33-476ab7cf18a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:28:09.400979 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-3be4e42b-08e5-4e8a-8d33-476ab7cf18a3 None None] MFA Rules not processed for user `f9ba1e56fd13424687ffd2e357835a76`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:28:09.402197 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-3be4e42b-08e5-4e8a-8d33-476ab7cf18a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:28:09.467380 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1440/2895] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 09:28:09 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 09:28:09.471612 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759abae3-c34c-4a21-8cf5-d8b3c57ef37c admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:28:09.535565 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759abae3-c34c-4a21-8cf5-d8b3c57ef37c admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:28:09.541039 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b094504b-6769-49f5-acbe-b08a3fef4b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:28:09.541642 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b094504b-6769-49f5-acbe-b08a3fef4b54 admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:28:09.541920 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b094504b-6769-49f5-acbe-b08a3fef4b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:28:09.542191 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b094504b-6769-49f5-acbe-b08a3fef4b54 admin admin] PATH_INFO: `/v3/projects` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:28:09.542808 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b094504b-6769-49f5-acbe-b08a3fef4b54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:28:09.544704 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b094504b-6769-49f5-acbe-b08a3fef4b54 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:28:09.631106 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1456/2896] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 09:28:09 2022] POST /identity/v3/projects => generated 383 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 09:28:09.635529 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be4e42b-08e5-4e8a-8d33-476ab7cf18a3 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:28:09.646372 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be4e42b-08e5-4e8a-8d33-476ab7cf18a3 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:28:09.663467 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:28:09.703573 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59b6f50-e868-4452-8353-8749fdb4ae5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:28:09.704116 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59b6f50-e868-4452-8353-8749fdb4ae5e admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:28:09.704564 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59b6f50-e868-4452-8353-8749fdb4ae5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:28:09.704877 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59b6f50-e868-4452-8353-8749fdb4ae5e admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:28:09.705954 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59b6f50-e868-4452-8353-8749fdb4ae5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:28:09.707855 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59b6f50-e868-4452-8353-8749fdb4ae5e admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:28:09.801021 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1441/2897] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 09:28:09 2022] POST /identity/v3/users => generated 298 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 09:28:09.805347 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b094504b-6769-49f5-acbe-b08a3fef4b54 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:28:09.815792 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b094504b-6769-49f5-acbe-b08a3fef4b54 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:28:09.820409 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef4b83b-670c-49ae-819d-ce69016c3dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:28:09.821027 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef4b83b-670c-49ae-819d-ce69016c3dd7 admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:28:09.821306 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef4b83b-670c-49ae-819d-ce69016c3dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:28:09.821571 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef4b83b-670c-49ae-819d-ce69016c3dd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:28:09.822202 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef4b83b-670c-49ae-819d-ce69016c3dd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:28:09.823786 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef4b83b-670c-49ae-819d-ce69016c3dd7 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:28:09.834213 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1457/2898] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 09:28:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 09:28:09.839864 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59b6f50-e868-4452-8353-8749fdb4ae5e admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:28:09.849634 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59b6f50-e868-4452-8353-8749fdb4ae5e admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:28:09.855050 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a8c1d4-2383-46cf-aca8-5ee60dbd0a10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:28:09.855683 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a8c1d4-2383-46cf-aca8-5ee60dbd0a10 admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:28:09.856005 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a8c1d4-2383-46cf-aca8-5ee60dbd0a10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:28:09.856273 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a8c1d4-2383-46cf-aca8-5ee60dbd0a10 admin admin] PATH_INFO: `/v3/projects/e3adbe04844c4b98b2741979133c12fa/users/5185617cf6d0405cacab672eecb772f4/roles/ec3751692d9c4ecf81b4a71c936c46be` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:28:09.870736 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a8c1d4-2383-46cf-aca8-5ee60dbd0a10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3adbe04844c4b98b2741979133c12fa, user_id=5185617cf6d0405cacab672eecb772f4, role_id=ec3751692d9c4ecf81b4a71c936c46be)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:28:09.873405 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a8c1d4-2383-46cf-aca8-5ee60dbd0a10 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:28:09.943380 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1442/2899] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 09:28:09 2022] PUT /identity/v3/projects/e3adbe04844c4b98b2741979133c12fa/users/5185617cf6d0405cacab672eecb772f4/roles/ec3751692d9c4ecf81b4a71c936c46be => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:28:09.947408 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef4b83b-670c-49ae-819d-ce69016c3dd7 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:28:09.957376 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef4b83b-670c-49ae-819d-ce69016c3dd7 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:28:10.015135 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc99734c-a4e8-438c-b56b-f4820c178b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:28:10.015732 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc99734c-a4e8-438c-b56b-f4820c178b4e admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:28:10.016010 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc99734c-a4e8-438c-b56b-f4820c178b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:28:10.016277 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc99734c-a4e8-438c-b56b-f4820c178b4e admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:28:10.016905 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc99734c-a4e8-438c-b56b-f4820c178b4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:28:10.018525 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc99734c-a4e8-438c-b56b-f4820c178b4e admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:28:10.029042 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1458/2900] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 09:28:09 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 09:28:10.033412 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a8c1d4-2383-46cf-aca8-5ee60dbd0a10 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:28:10.044272 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a8c1d4-2383-46cf-aca8-5ee60dbd0a10 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:28:10.048886 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902cba1a-25b9-4dae-8692-4ab04dbae121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:28:10.049560 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902cba1a-25b9-4dae-8692-4ab04dbae121 admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:28:10.049843 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902cba1a-25b9-4dae-8692-4ab04dbae121 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:28:10.050321 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902cba1a-25b9-4dae-8692-4ab04dbae121 admin admin] PATH_INFO: `/v3/projects/e3adbe04844c4b98b2741979133c12fa/users/5185617cf6d0405cacab672eecb772f4/roles/ec3751692d9c4ecf81b4a71c936c46be` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:28:10.052269 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902cba1a-25b9-4dae-8692-4ab04dbae121 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3adbe04844c4b98b2741979133c12fa, user_id=5185617cf6d0405cacab672eecb772f4, role_id=ec3751692d9c4ecf81b4a71c936c46be)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:28:10.054940 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902cba1a-25b9-4dae-8692-4ab04dbae121 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:28:10.065392 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1443/2901] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 09:28:10 2022] PUT /identity/v3/projects/e3adbe04844c4b98b2741979133c12fa/users/5185617cf6d0405cacab672eecb772f4/roles/ec3751692d9c4ecf81b4a71c936c46be => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:28:10.076303 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afe8156-8c38-44ac-81d8-63d182bfb249 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:28:10.076614 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afe8156-8c38-44ac-81d8-63d182bfb249 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:28:10.076888 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afe8156-8c38-44ac-81d8-63d182bfb249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:28:10.096617 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:28:10.104785 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-5afe8156-8c38-44ac-81d8-63d182bfb249 None None] MFA Rules not processed for user `5185617cf6d0405cacab672eecb772f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:28:10.106037 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-5afe8156-8c38-44ac-81d8-63d182bfb249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:28:10.173395 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1459/2902] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 09:28:10 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:28:10.184299 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902cba1a-25b9-4dae-8692-4ab04dbae121 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:28:10.186009 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-902cba1a-25b9-4dae-8692-4ab04dbae121 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:28:10.250518 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902cba1a-25b9-4dae-8692-4ab04dbae121 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:28:10.257615 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dfc6e3-6fcd-4e39-ba9c-48906ca10908 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ba74e28ced4cbba857f54222dfbe41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:28:10.258591 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dfc6e3-6fcd-4e39-ba9c-48906ca10908 service cinder] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:28:10.259034 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dfc6e3-6fcd-4e39-ba9c-48906ca10908 service cinder] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:28:10.259482 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dfc6e3-6fcd-4e39-ba9c-48906ca10908 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:28:10.329679 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dfc6e3-6fcd-4e39-ba9c-48906ca10908 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:28:10.331091 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dfc6e3-6fcd-4e39-ba9c-48906ca10908 service cinder] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:28:10.341022 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1444/2903] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:28:10 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:28:16.618920 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afe8156-8c38-44ac-81d8-63d182bfb249 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:28:16.623193 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-5afe8156-8c38-44ac-81d8-63d182bfb249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:28:16.632574 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afe8156-8c38-44ac-81d8-63d182bfb249 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:28:16.636135 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11b8a68-667e-4f78-be21-ea50ad01baee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ba74e28ced4cbba857f54222dfbe41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:28:16.636725 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11b8a68-667e-4f78-be21-ea50ad01baee service cinder] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:28:16.637048 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11b8a68-667e-4f78-be21-ea50ad01baee service cinder] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:28:16.637322 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11b8a68-667e-4f78-be21-ea50ad01baee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:28:16.646596 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11b8a68-667e-4f78-be21-ea50ad01baee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:28:16.651905 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11b8a68-667e-4f78-be21-ea50ad01baee service cinder] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:28:16.658490 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1460/2904] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:28:16 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:28:17.777600 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dfc6e3-6fcd-4e39-ba9c-48906ca10908 service cinder] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:28:17.781590 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-48dfc6e3-6fcd-4e39-ba9c-48906ca10908 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:28:17.810794 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:28:17.845031 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dfc6e3-6fcd-4e39-ba9c-48906ca10908 service cinder] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:28:17.849555 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5e305a-2bf4-4b11-89ae-b1f7911d00ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:28:17.850177 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5e305a-2bf4-4b11-89ae-b1f7911d00ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:28:17.852885 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5e305a-2bf4-4b11-89ae-b1f7911d00ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:28:17.852885 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5e305a-2bf4-4b11-89ae-b1f7911d00ea admin admin] PATH_INFO: `/v3/users/3d176cb1bc66423b8647a5a76efc9344` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:28:17.852885 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5e305a-2bf4-4b11-89ae-b1f7911d00ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d176cb1bc66423b8647a5a76efc9344)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:28:17.853786 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5e305a-2bf4-4b11-89ae-b1f7911d00ea admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:28:18.034685 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-3c5e305a-2bf4-4b11-89ae-b1f7911d00ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d176cb1bc66423b8647a5a76efc9344'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:28:18.047501 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-3c5e305a-2bf4-4b11-89ae-b1f7911d00ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d176cb1bc66423b8647a5a76efc9344'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:28:18.055232 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-3c5e305a-2bf4-4b11-89ae-b1f7911d00ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d176cb1bc66423b8647a5a76efc9344'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:28:18.055956 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-3c5e305a-2bf4-4b11-89ae-b1f7911d00ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d176cb1bc66423b8647a5a76efc9344'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:28:18.056572 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-3c5e305a-2bf4-4b11-89ae-b1f7911d00ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d176cb1bc66423b8647a5a76efc9344'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:28:18.142310 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1445/2905] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 09:28:17 2022] DELETE /identity/v3/users/3d176cb1bc66423b8647a5a76efc9344 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:28:18.395858 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11b8a68-667e-4f78-be21-ea50ad01baee service cinder] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:28:18.408089 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11b8a68-667e-4f78-be21-ea50ad01baee service cinder] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:28:18.470779 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a2242a-f1d9-404c-85fb-6ea61bafc017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:28:18.471515 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a2242a-f1d9-404c-85fb-6ea61bafc017 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:28:18.471800 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a2242a-f1d9-404c-85fb-6ea61bafc017 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:28:18.472075 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a2242a-f1d9-404c-85fb-6ea61bafc017 admin admin] PATH_INFO: `/v3/projects/25372be0a838467aab413517420d3164` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:28:18.473227 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a2242a-f1d9-404c-85fb-6ea61bafc017 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25372be0a838467aab413517420d3164)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:28:18.475175 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a2242a-f1d9-404c-85fb-6ea61bafc017 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:28:18.572439 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-e1a2242a-f1d9-404c-85fb-6ea61bafc017 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25372be0a838467aab413517420d3164', 'request_id': 'req-e1a2242a-f1d9-404c-85fb-6ea61bafc017'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:28:18.577736 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-e1a2242a-f1d9-404c-85fb-6ea61bafc017 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25372be0a838467aab413517420d3164', 'request_id': 'req-e1a2242a-f1d9-404c-85fb-6ea61bafc017'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:28:18.639455 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-e1a2242a-f1d9-404c-85fb-6ea61bafc017 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25372be0a838467aab413517420d3164', 'request_id': 'req-e1a2242a-f1d9-404c-85fb-6ea61bafc017'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:28:18.651272 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-e1a2242a-f1d9-404c-85fb-6ea61bafc017 admin admin] The token cache is being invalidate because project 25372be0a838467aab413517420d3164 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 09:28:18.651556 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-e1a2242a-f1d9-404c-85fb-6ea61bafc017 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:28:18.652175 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-e1a2242a-f1d9-404c-85fb-6ea61bafc017 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:28:18.653696 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1461/2906] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 09:28:18 2022] DELETE /identity/v3/projects/25372be0a838467aab413517420d3164 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:28:18.658539 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad23bcb0-a4bb-4e55-b1e5-988522544142 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:28:18.658824 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad23bcb0-a4bb-4e55-b1e5-988522544142 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:28:18.659100 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad23bcb0-a4bb-4e55-b1e5-988522544142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:28:18.681221 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-ad23bcb0-a4bb-4e55-b1e5-988522544142 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:28:18.682684 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-ad23bcb0-a4bb-4e55-b1e5-988522544142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:28:18.743605 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1446/2907] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 09:28:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:28:18.751568 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d810501b-f3eb-4a09-a1dc-1a825b94de0b None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:28:18.752290 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d810501b-f3eb-4a09-a1dc-1a825b94de0b None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:28:18.752290 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d810501b-f3eb-4a09-a1dc-1a825b94de0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:28:18.772306 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-d810501b-f3eb-4a09-a1dc-1a825b94de0b None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:28:18.774244 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-d810501b-f3eb-4a09-a1dc-1a825b94de0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:28:18.782224 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1462/2908] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 09:28:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:28:18.786975 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad23bcb0-a4bb-4e55-b1e5-988522544142 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:28:18.796500 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad23bcb0-a4bb-4e55-b1e5-988522544142 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:28:18.801162 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c829ffc-becc-49c4-b65e-cf855dcf6922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:28:18.801751 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c829ffc-becc-49c4-b65e-cf855dcf6922 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:28:18.802031 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c829ffc-becc-49c4-b65e-cf855dcf6922 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:28:18.802302 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c829ffc-becc-49c4-b65e-cf855dcf6922 admin admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:28:18.802938 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c829ffc-becc-49c4-b65e-cf855dcf6922 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:28:18.803245 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c829ffc-becc-49c4-b65e-cf855dcf6922 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:28:18.804861 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c829ffc-becc-49c4-b65e-cf855dcf6922 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:28:18.816739 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1447/2909] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Aug 10 09:28:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:28:18.821898 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d810501b-f3eb-4a09-a1dc-1a825b94de0b None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:28:18.827570 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:28:18.831308 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d810501b-f3eb-4a09-a1dc-1a825b94de0b None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:28:18.836768 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4163118-0fc7-47af-842a-240cf39db46a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:28:18.837395 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4163118-0fc7-47af-842a-240cf39db46a admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:28:18.837674 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4163118-0fc7-47af-842a-240cf39db46a admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:28:18.837946 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4163118-0fc7-47af-842a-240cf39db46a admin admin] PATH_INFO: `/v3/projects` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:28:18.838548 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4163118-0fc7-47af-842a-240cf39db46a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:28:18.840433 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4163118-0fc7-47af-842a-240cf39db46a admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:28:18.947349 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1463/2910] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 09:28:18 2022] POST /identity/v3/projects => generated 391 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 09:28:18.948853 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c829ffc-becc-49c4-b65e-cf855dcf6922 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:28:18.959552 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c829ffc-becc-49c4-b65e-cf855dcf6922 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:28:18.990043 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:28:19.015963 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e25113f-e489-4b54-a0d5-993a0176a013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:28:19.016570 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e25113f-e489-4b54-a0d5-993a0176a013 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:28:19.016845 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e25113f-e489-4b54-a0d5-993a0176a013 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:28:19.017150 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e25113f-e489-4b54-a0d5-993a0176a013 admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:28:19.017784 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e25113f-e489-4b54-a0d5-993a0176a013 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:28:19.019796 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e25113f-e489-4b54-a0d5-993a0176a013 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:28:19.105273 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1448/2911] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 09:28:18 2022] POST /identity/v3/users => generated 296 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 09:28:19.108538 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4163118-0fc7-47af-842a-240cf39db46a admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:28:19.119550 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4163118-0fc7-47af-842a-240cf39db46a admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:28:19.124096 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45516c52-191e-41d7-b45a-43c716776b7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:28:19.124687 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45516c52-191e-41d7-b45a-43c716776b7b admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:28:19.124991 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45516c52-191e-41d7-b45a-43c716776b7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:28:19.125263 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45516c52-191e-41d7-b45a-43c716776b7b admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:28:19.125897 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45516c52-191e-41d7-b45a-43c716776b7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:28:19.127467 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45516c52-191e-41d7-b45a-43c716776b7b admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:28:19.138241 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1464/2912] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 09:28:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 09:28:19.142291 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e25113f-e489-4b54-a0d5-993a0176a013 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:28:19.152555 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e25113f-e489-4b54-a0d5-993a0176a013 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:28:19.157591 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cba8fc-aa14-4af4-a97a-8af0f4922e07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:28:19.160254 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cba8fc-aa14-4af4-a97a-8af0f4922e07 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:28:19.160254 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cba8fc-aa14-4af4-a97a-8af0f4922e07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:28:19.160254 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cba8fc-aa14-4af4-a97a-8af0f4922e07 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:28:19.160254 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cba8fc-aa14-4af4-a97a-8af0f4922e07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:28:19.161115 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cba8fc-aa14-4af4-a97a-8af0f4922e07 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:28:19.172305 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1449/2913] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 09:28:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 09:28:19.176566 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45516c52-191e-41d7-b45a-43c716776b7b admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:28:19.188085 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45516c52-191e-41d7-b45a-43c716776b7b admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:28:19.192951 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd4ca43-762f-49e6-827f-ea3c75df19f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:28:19.193607 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd4ca43-762f-49e6-827f-ea3c75df19f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:28:19.193886 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd4ca43-762f-49e6-827f-ea3c75df19f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:28:19.194183 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd4ca43-762f-49e6-827f-ea3c75df19f1 admin admin] PATH_INFO: `/v3/projects/9480b2756d1840d68346c9f18e6e92f9/users/cf1c2b6ed163403a9a2a72ba047f97e0/roles/44b64db484f24a46bf0b7ced46b519bf` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:28:19.209268 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd4ca43-762f-49e6-827f-ea3c75df19f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9480b2756d1840d68346c9f18e6e92f9, user_id=cf1c2b6ed163403a9a2a72ba047f97e0, role_id=44b64db484f24a46bf0b7ced46b519bf)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:28:19.212007 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd4ca43-762f-49e6-827f-ea3c75df19f1 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:28:19.311182 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1465/2914] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 09:28:19 2022] PUT /identity/v3/projects/9480b2756d1840d68346c9f18e6e92f9/users/cf1c2b6ed163403a9a2a72ba047f97e0/roles/44b64db484f24a46bf0b7ced46b519bf => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 09:28:19.322380 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9f1d12-c1d6-417a-b1b2-20180cbf8a32 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:28:19.326040 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9f1d12-c1d6-417a-b1b2-20180cbf8a32 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:28:19.326040 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9f1d12-c1d6-417a-b1b2-20180cbf8a32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:28:19.350470 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-8e9f1d12-c1d6-417a-b1b2-20180cbf8a32 None None] MFA Rules not processed for user `cf1c2b6ed163403a9a2a72ba047f97e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:28:19.351908 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-8e9f1d12-c1d6-417a-b1b2-20180cbf8a32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:28:19.416564 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1450/2915] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 09:28:19 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:28:19.427518 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd4ca43-762f-49e6-827f-ea3c75df19f1 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:28:19.429904 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-dcd4ca43-762f-49e6-827f-ea3c75df19f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:28:19.489199 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd4ca43-762f-49e6-827f-ea3c75df19f1 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:28:19.493475 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a522180-4ba1-4538-a19e-4f9f0ad7be09 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ba74e28ced4cbba857f54222dfbe41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:28:19.494079 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a522180-4ba1-4538-a19e-4f9f0ad7be09 service cinder] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:28:19.494359 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a522180-4ba1-4538-a19e-4f9f0ad7be09 service cinder] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:28:19.494627 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a522180-4ba1-4538-a19e-4f9f0ad7be09 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:28:19.502932 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a522180-4ba1-4538-a19e-4f9f0ad7be09 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:28:19.504360 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a522180-4ba1-4538-a19e-4f9f0ad7be09 service cinder] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:28:19.513719 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1466/2916] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:28:19 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:28:22.484808 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9f1d12-c1d6-417a-b1b2-20180cbf8a32 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:28:22.486164 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-8e9f1d12-c1d6-417a-b1b2-20180cbf8a32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:28:22.550672 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9f1d12-c1d6-417a-b1b2-20180cbf8a32 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:28:22.555874 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d5bd3a-87c0-4bea-ae94-5b4302366bdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:28:22.556509 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d5bd3a-87c0-4bea-ae94-5b4302366bdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:28:22.556772 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d5bd3a-87c0-4bea-ae94-5b4302366bdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:28:22.557080 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d5bd3a-87c0-4bea-ae94-5b4302366bdb admin admin] PATH_INFO: `/v3/users/f9ba1e56fd13424687ffd2e357835a76` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:28:22.558460 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d5bd3a-87c0-4bea-ae94-5b4302366bdb admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9ba1e56fd13424687ffd2e357835a76)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:28:22.560478 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d5bd3a-87c0-4bea-ae94-5b4302366bdb admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:28:22.778834 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:28:22.781457 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-80d5bd3a-87c0-4bea-ae94-5b4302366bdb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9ba1e56fd13424687ffd2e357835a76'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:28:22.793255 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-80d5bd3a-87c0-4bea-ae94-5b4302366bdb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9ba1e56fd13424687ffd2e357835a76'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:28:22.800033 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-80d5bd3a-87c0-4bea-ae94-5b4302366bdb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9ba1e56fd13424687ffd2e357835a76'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:28:22.800785 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-80d5bd3a-87c0-4bea-ae94-5b4302366bdb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9ba1e56fd13424687ffd2e357835a76'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:28:22.801422 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-80d5bd3a-87c0-4bea-ae94-5b4302366bdb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9ba1e56fd13424687ffd2e357835a76'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:28:22.926192 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1451/2917] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 09:28:22 2022] DELETE /identity/v3/users/f9ba1e56fd13424687ffd2e357835a76 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:28:22.931415 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9624439a-4453-45a5-8f8b-66b6fba5d5bb None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:28:22.932014 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9624439a-4453-45a5-8f8b-66b6fba5d5bb None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:28:22.932014 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9624439a-4453-45a5-8f8b-66b6fba5d5bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:28:22.951413 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-9624439a-4453-45a5-8f8b-66b6fba5d5bb None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:28:22.954527 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-9624439a-4453-45a5-8f8b-66b6fba5d5bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:28:23.009945 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1467/2918] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 09:28:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:28:23.019644 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d5bd3a-87c0-4bea-ae94-5b4302366bdb admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:28:23.021057 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-80d5bd3a-87c0-4bea-ae94-5b4302366bdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:28:23.078121 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d5bd3a-87c0-4bea-ae94-5b4302366bdb admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:28:23.081842 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c0d8d8-2599-4fc3-aaaa-d9fd114baa44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:28:23.082440 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c0d8d8-2599-4fc3-aaaa-d9fd114baa44 service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:28:23.082722 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c0d8d8-2599-4fc3-aaaa-d9fd114baa44 service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:28:23.082991 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c0d8d8-2599-4fc3-aaaa-d9fd114baa44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:28:23.092119 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c0d8d8-2599-4fc3-aaaa-d9fd114baa44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:28:23.093578 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c0d8d8-2599-4fc3-aaaa-d9fd114baa44 service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:28:23.103033 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1452/2919] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:28:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 09:28:23.529462 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9624439a-4453-45a5-8f8b-66b6fba5d5bb None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:28:23.530893 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-9624439a-4453-45a5-8f8b-66b6fba5d5bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:28:23.543215 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9624439a-4453-45a5-8f8b-66b6fba5d5bb None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:28:23.547954 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07e2fcb-5a68-4237-ad14-95d57081e5cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:28:23.548617 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07e2fcb-5a68-4237-ad14-95d57081e5cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:28:23.548899 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07e2fcb-5a68-4237-ad14-95d57081e5cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:28:23.549199 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07e2fcb-5a68-4237-ad14-95d57081e5cd admin admin] PATH_INFO: `/v3/projects/cd7626373c8440acbed250fbaf2b5134` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:28:23.550437 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07e2fcb-5a68-4237-ad14-95d57081e5cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd7626373c8440acbed250fbaf2b5134)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:28:23.552398 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07e2fcb-5a68-4237-ad14-95d57081e5cd admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:28:23.656690 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:28:23.659683 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-d07e2fcb-5a68-4237-ad14-95d57081e5cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd7626373c8440acbed250fbaf2b5134', 'request_id': 'req-d07e2fcb-5a68-4237-ad14-95d57081e5cd'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:28:23.665302 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-d07e2fcb-5a68-4237-ad14-95d57081e5cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd7626373c8440acbed250fbaf2b5134', 'request_id': 'req-d07e2fcb-5a68-4237-ad14-95d57081e5cd'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:28:23.738207 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-d07e2fcb-5a68-4237-ad14-95d57081e5cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd7626373c8440acbed250fbaf2b5134', 'request_id': 'req-d07e2fcb-5a68-4237-ad14-95d57081e5cd'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:28:23.750385 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-d07e2fcb-5a68-4237-ad14-95d57081e5cd admin admin] The token cache is being invalidate because project cd7626373c8440acbed250fbaf2b5134 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 09:28:23.750676 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-d07e2fcb-5a68-4237-ad14-95d57081e5cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:28:23.751325 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-d07e2fcb-5a68-4237-ad14-95d57081e5cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:28:23.752830 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1468/2920] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 09:28:23 2022] DELETE /identity/v3/projects/cd7626373c8440acbed250fbaf2b5134 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:28:23.757144 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c0d8d8-2599-4fc3-aaaa-d9fd114baa44 service neutron] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:28:23.767804 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c0d8d8-2599-4fc3-aaaa-d9fd114baa44 service neutron] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:28:23.825681 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a71074-d96d-4692-912c-be903df6f958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:28:23.826234 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a71074-d96d-4692-912c-be903df6f958 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:28:23.826510 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a71074-d96d-4692-912c-be903df6f958 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:28:23.826780 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a71074-d96d-4692-912c-be903df6f958 admin admin] PATH_INFO: `/v3/users/5185617cf6d0405cacab672eecb772f4` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:28:23.827908 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a71074-d96d-4692-912c-be903df6f958 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5185617cf6d0405cacab672eecb772f4)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:28:23.829853 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a71074-d96d-4692-912c-be903df6f958 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:28:24.035819 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-56a71074-d96d-4692-912c-be903df6f958 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5185617cf6d0405cacab672eecb772f4'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:28:24.047996 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-56a71074-d96d-4692-912c-be903df6f958 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5185617cf6d0405cacab672eecb772f4'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:28:24.049168 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:28:24.055870 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-56a71074-d96d-4692-912c-be903df6f958 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5185617cf6d0405cacab672eecb772f4'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:28:24.056556 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-56a71074-d96d-4692-912c-be903df6f958 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5185617cf6d0405cacab672eecb772f4'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:28:24.057210 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-56a71074-d96d-4692-912c-be903df6f958 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5185617cf6d0405cacab672eecb772f4'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:28:24.136562 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1453/2921] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 09:28:23 2022] DELETE /identity/v3/users/5185617cf6d0405cacab672eecb772f4 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:28:24.540735 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07e2fcb-5a68-4237-ad14-95d57081e5cd admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:28:24.543526 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-d07e2fcb-5a68-4237-ad14-95d57081e5cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:28:24.556530 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07e2fcb-5a68-4237-ad14-95d57081e5cd admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:28:24.614772 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6fe131-2303-465b-b724-bcd49b5ac02d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:28:24.615405 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6fe131-2303-465b-b724-bcd49b5ac02d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:28:24.615682 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6fe131-2303-465b-b724-bcd49b5ac02d admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:28:24.615955 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6fe131-2303-465b-b724-bcd49b5ac02d admin admin] PATH_INFO: `/v3/projects/e3adbe04844c4b98b2741979133c12fa` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:28:24.617096 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6fe131-2303-465b-b724-bcd49b5ac02d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3adbe04844c4b98b2741979133c12fa)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:28:24.618999 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6fe131-2303-465b-b724-bcd49b5ac02d admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:28:24.732595 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-cc6fe131-2303-465b-b724-bcd49b5ac02d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3adbe04844c4b98b2741979133c12fa', 'request_id': 'req-cc6fe131-2303-465b-b724-bcd49b5ac02d'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:28:24.737799 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-cc6fe131-2303-465b-b724-bcd49b5ac02d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3adbe04844c4b98b2741979133c12fa', 'request_id': 'req-cc6fe131-2303-465b-b724-bcd49b5ac02d'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:28:24.795331 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-cc6fe131-2303-465b-b724-bcd49b5ac02d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3adbe04844c4b98b2741979133c12fa', 'request_id': 'req-cc6fe131-2303-465b-b724-bcd49b5ac02d'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:28:24.807347 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-cc6fe131-2303-465b-b724-bcd49b5ac02d admin admin] The token cache is being invalidate because project e3adbe04844c4b98b2741979133c12fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 09:28:24.807916 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-cc6fe131-2303-465b-b724-bcd49b5ac02d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:28:24.808563 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-cc6fe131-2303-465b-b724-bcd49b5ac02d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:28:24.810067 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1469/2922] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 09:28:24 2022] DELETE /identity/v3/projects/e3adbe04844c4b98b2741979133c12fa => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:28:24.815971 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0425b67f-3311-4ae0-8c30-05c280571527 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:28:24.816434 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0425b67f-3311-4ae0-8c30-05c280571527 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:28:24.816553 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0425b67f-3311-4ae0-8c30-05c280571527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:28:24.836397 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-0425b67f-3311-4ae0-8c30-05c280571527 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:28:24.838741 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-0425b67f-3311-4ae0-8c30-05c280571527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:28:24.896640 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1454/2923] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 09:28:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 09:28:24.905812 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e8b773-3b3e-40b6-b569-c036ecd7e074 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:28:24.906096 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e8b773-3b3e-40b6-b569-c036ecd7e074 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:28:24.906368 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e8b773-3b3e-40b6-b569-c036ecd7e074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:28:24.925933 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-21e8b773-3b3e-40b6-b569-c036ecd7e074 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:28:24.927070 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-21e8b773-3b3e-40b6-b569-c036ecd7e074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:28:24.935189 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1470/2924] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 09:28:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:28:24.939395 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0425b67f-3311-4ae0-8c30-05c280571527 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:28:24.949617 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0425b67f-3311-4ae0-8c30-05c280571527 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:28:24.954299 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d588f505-3133-4d79-b8c1-8b70c7fcd37d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:28:24.954921 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d588f505-3133-4d79-b8c1-8b70c7fcd37d admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:28:24.955199 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d588f505-3133-4d79-b8c1-8b70c7fcd37d admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:28:24.955467 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d588f505-3133-4d79-b8c1-8b70c7fcd37d admin admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:28:24.956110 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d588f505-3133-4d79-b8c1-8b70c7fcd37d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:28:24.956412 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d588f505-3133-4d79-b8c1-8b70c7fcd37d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:28:24.958083 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d588f505-3133-4d79-b8c1-8b70c7fcd37d admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:28:24.970152 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1455/2925] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Aug 10 09:28:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:28:24.974439 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e8b773-3b3e-40b6-b569-c036ecd7e074 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:28:24.983973 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e8b773-3b3e-40b6-b569-c036ecd7e074 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:28:24.988535 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1e5f0d-ab05-4600-b9dd-70120a5581b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:28:24.989154 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1e5f0d-ab05-4600-b9dd-70120a5581b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:28:24.989429 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1e5f0d-ab05-4600-b9dd-70120a5581b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:28:24.989696 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1e5f0d-ab05-4600-b9dd-70120a5581b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:28:24.990304 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1e5f0d-ab05-4600-b9dd-70120a5581b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:28:24.992194 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1e5f0d-ab05-4600-b9dd-70120a5581b2 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:28:25.071987 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1471/2926] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 09:28:24 2022] POST /identity/v3/projects => generated 377 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 09:28:25.076342 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d588f505-3133-4d79-b8c1-8b70c7fcd37d admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:28:25.086428 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d588f505-3133-4d79-b8c1-8b70c7fcd37d admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:28:25.141799 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ca8a35-bc52-4ef4-a7cc-daaeddcc7810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:28:25.142395 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ca8a35-bc52-4ef4-a7cc-daaeddcc7810 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:28:25.142676 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ca8a35-bc52-4ef4-a7cc-daaeddcc7810 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:28:25.142945 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ca8a35-bc52-4ef4-a7cc-daaeddcc7810 admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:28:25.145265 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ca8a35-bc52-4ef4-a7cc-daaeddcc7810 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:28:25.145521 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ca8a35-bc52-4ef4-a7cc-daaeddcc7810 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:28:25.239744 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1456/2927] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 09:28:25 2022] POST /identity/v3/users => generated 289 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 09:28:25.244937 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1e5f0d-ab05-4600-b9dd-70120a5581b2 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:28:25.255352 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1e5f0d-ab05-4600-b9dd-70120a5581b2 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:28:25.260962 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e1ca49-f0bc-4ba3-a766-cd3cb6f2f524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:28:25.261565 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e1ca49-f0bc-4ba3-a766-cd3cb6f2f524 admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:28:25.261846 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e1ca49-f0bc-4ba3-a766-cd3cb6f2f524 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:28:25.262120 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e1ca49-f0bc-4ba3-a766-cd3cb6f2f524 admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:28:25.262759 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e1ca49-f0bc-4ba3-a766-cd3cb6f2f524 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:28:25.264359 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e1ca49-f0bc-4ba3-a766-cd3cb6f2f524 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:28:25.274149 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1472/2928] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 09:28:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 09:28:25.278013 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ca8a35-bc52-4ef4-a7cc-daaeddcc7810 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:28:25.288006 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ca8a35-bc52-4ef4-a7cc-daaeddcc7810 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:28:25.293238 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a01730-72c9-4da2-bc26-3a93ebe8c758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:28:25.293823 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a01730-72c9-4da2-bc26-3a93ebe8c758 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:28:25.294102 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a01730-72c9-4da2-bc26-3a93ebe8c758 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:28:25.294373 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a01730-72c9-4da2-bc26-3a93ebe8c758 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:28:25.295006 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a01730-72c9-4da2-bc26-3a93ebe8c758 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:28:25.296582 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a01730-72c9-4da2-bc26-3a93ebe8c758 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:28:25.310127 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1457/2929] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 09:28:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 09:28:25.314319 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e1ca49-f0bc-4ba3-a766-cd3cb6f2f524 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:28:25.327403 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e1ca49-f0bc-4ba3-a766-cd3cb6f2f524 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:28:25.331975 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce6dee1-29a7-4d54-806a-a2f35cf8a6fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:28:25.332617 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce6dee1-29a7-4d54-806a-a2f35cf8a6fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:28:25.332938 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce6dee1-29a7-4d54-806a-a2f35cf8a6fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:28:25.333222 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce6dee1-29a7-4d54-806a-a2f35cf8a6fb admin admin] PATH_INFO: `/v3/projects/902c69449fff4884b9ca0d955b105922/users/31ccfce2e43f498ab30e5d18f89e3d22/roles/44b64db484f24a46bf0b7ced46b519bf` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:28:25.347432 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce6dee1-29a7-4d54-806a-a2f35cf8a6fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=902c69449fff4884b9ca0d955b105922, user_id=31ccfce2e43f498ab30e5d18f89e3d22, role_id=44b64db484f24a46bf0b7ced46b519bf)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:28:25.350127 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce6dee1-29a7-4d54-806a-a2f35cf8a6fb admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:28:25.408809 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:28:25.413648 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1473/2930] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 09:28:25 2022] PUT /identity/v3/projects/902c69449fff4884b9ca0d955b105922/users/31ccfce2e43f498ab30e5d18f89e3d22/roles/44b64db484f24a46bf0b7ced46b519bf => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 09:28:25.426500 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e47833-6ca8-4c97-9da9-709eb7d737da None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:28:25.426789 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e47833-6ca8-4c97-9da9-709eb7d737da None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:28:25.427091 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e47833-6ca8-4c97-9da9-709eb7d737da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:28:25.455990 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-55e47833-6ca8-4c97-9da9-709eb7d737da None None] MFA Rules not processed for user `31ccfce2e43f498ab30e5d18f89e3d22`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:28:25.457290 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-55e47833-6ca8-4c97-9da9-709eb7d737da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:28:25.496954 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:28:25.521774 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1458/2931] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 09:28:25 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:28:25.533525 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce6dee1-29a7-4d54-806a-a2f35cf8a6fb admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:28:25.535360 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-dce6dee1-29a7-4d54-806a-a2f35cf8a6fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:28:25.592804 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce6dee1-29a7-4d54-806a-a2f35cf8a6fb admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:28:25.596529 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4da9bd1-0c84-4c7a-8878-01cc230cc983 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ba74e28ced4cbba857f54222dfbe41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:28:25.597150 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4da9bd1-0c84-4c7a-8878-01cc230cc983 service cinder] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:28:25.597433 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4da9bd1-0c84-4c7a-8878-01cc230cc983 service cinder] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:28:25.597706 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4da9bd1-0c84-4c7a-8878-01cc230cc983 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:28:25.606708 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4da9bd1-0c84-4c7a-8878-01cc230cc983 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:28:25.608132 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4da9bd1-0c84-4c7a-8878-01cc230cc983 service cinder] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:28:25.616979 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1474/2932] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:28:25 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:28:35.610851 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=10 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:28:35.612884 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=26 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:28:51.506900 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e47833-6ca8-4c97-9da9-709eb7d737da None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:28:51.508363 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-55e47833-6ca8-4c97-9da9-709eb7d737da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:28:51.573200 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e47833-6ca8-4c97-9da9-709eb7d737da None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:28:51.578184 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7c778d-0851-4da4-8d04-c6d49fcf73c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:28:51.578829 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7c778d-0851-4da4-8d04-c6d49fcf73c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:28:51.579085 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7c778d-0851-4da4-8d04-c6d49fcf73c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:28:51.579352 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7c778d-0851-4da4-8d04-c6d49fcf73c4 admin admin] PATH_INFO: `/v3/users/cf1c2b6ed163403a9a2a72ba047f97e0` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:28:51.580451 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7c778d-0851-4da4-8d04-c6d49fcf73c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf1c2b6ed163403a9a2a72ba047f97e0)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:28:51.582389 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7c778d-0851-4da4-8d04-c6d49fcf73c4 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:28:51.792971 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-5f7c778d-0851-4da4-8d04-c6d49fcf73c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf1c2b6ed163403a9a2a72ba047f97e0'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:28:51.806072 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-5f7c778d-0851-4da4-8d04-c6d49fcf73c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf1c2b6ed163403a9a2a72ba047f97e0'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:28:51.816509 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-5f7c778d-0851-4da4-8d04-c6d49fcf73c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf1c2b6ed163403a9a2a72ba047f97e0'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:28:51.816509 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-5f7c778d-0851-4da4-8d04-c6d49fcf73c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf1c2b6ed163403a9a2a72ba047f97e0'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:28:51.816509 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-5f7c778d-0851-4da4-8d04-c6d49fcf73c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf1c2b6ed163403a9a2a72ba047f97e0'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:28:51.967169 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1459/2933] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 09:28:51 2022] DELETE /identity/v3/users/cf1c2b6ed163403a9a2a72ba047f97e0 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:28:51.971804 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b4a4bb-7a42-4352-ab79-0b23ddd120a7 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:28:51.972324 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b4a4bb-7a42-4352-ab79-0b23ddd120a7 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:28:51.972441 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b4a4bb-7a42-4352-ab79-0b23ddd120a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:28:51.992368 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-d6b4a4bb-7a42-4352-ab79-0b23ddd120a7 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:28:51.993610 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-d6b4a4bb-7a42-4352-ab79-0b23ddd120a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:28:52.052480 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1475/2934] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 09:28:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:28:52.063208 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7c778d-0851-4da4-8d04-c6d49fcf73c4 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:28:52.064692 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-5f7c778d-0851-4da4-8d04-c6d49fcf73c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:28:52.120998 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7c778d-0851-4da4-8d04-c6d49fcf73c4 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:28:52.125111 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c170e634-95d4-47bf-947a-0526b8a5b56c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:28:52.125756 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c170e634-95d4-47bf-947a-0526b8a5b56c service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:28:52.126037 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c170e634-95d4-47bf-947a-0526b8a5b56c service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:28:52.126307 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c170e634-95d4-47bf-947a-0526b8a5b56c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:28:52.135289 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c170e634-95d4-47bf-947a-0526b8a5b56c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:28:52.136715 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c170e634-95d4-47bf-947a-0526b8a5b56c service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:28:52.146767 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1460/2935] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:28:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:28:52.555763 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b4a4bb-7a42-4352-ab79-0b23ddd120a7 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:28:52.557224 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-d6b4a4bb-7a42-4352-ab79-0b23ddd120a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:28:52.570223 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b4a4bb-7a42-4352-ab79-0b23ddd120a7 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:28:52.574940 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bc2f68-73c0-4f26-9ca7-08336f9ff1e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:28:52.575616 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bc2f68-73c0-4f26-9ca7-08336f9ff1e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:28:52.575854 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bc2f68-73c0-4f26-9ca7-08336f9ff1e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:28:52.576130 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bc2f68-73c0-4f26-9ca7-08336f9ff1e5 admin admin] PATH_INFO: `/v3/projects/9480b2756d1840d68346c9f18e6e92f9` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:28:52.577954 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bc2f68-73c0-4f26-9ca7-08336f9ff1e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9480b2756d1840d68346c9f18e6e92f9)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:28:52.579972 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bc2f68-73c0-4f26-9ca7-08336f9ff1e5 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:28:52.688063 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-e8bc2f68-73c0-4f26-9ca7-08336f9ff1e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9480b2756d1840d68346c9f18e6e92f9', 'request_id': 'req-e8bc2f68-73c0-4f26-9ca7-08336f9ff1e5'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:28:52.693551 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-e8bc2f68-73c0-4f26-9ca7-08336f9ff1e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9480b2756d1840d68346c9f18e6e92f9', 'request_id': 'req-e8bc2f68-73c0-4f26-9ca7-08336f9ff1e5'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:28:52.766776 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-e8bc2f68-73c0-4f26-9ca7-08336f9ff1e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9480b2756d1840d68346c9f18e6e92f9', 'request_id': 'req-e8bc2f68-73c0-4f26-9ca7-08336f9ff1e5'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:28:52.778043 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-e8bc2f68-73c0-4f26-9ca7-08336f9ff1e5 admin admin] The token cache is being invalidate because project 9480b2756d1840d68346c9f18e6e92f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 09:28:52.778332 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-e8bc2f68-73c0-4f26-9ca7-08336f9ff1e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:28:52.778982 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-e8bc2f68-73c0-4f26-9ca7-08336f9ff1e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:28:52.781017 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1476/2936] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 09:28:52 2022] DELETE /identity/v3/projects/9480b2756d1840d68346c9f18e6e92f9 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:29:01.957132 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c170e634-95d4-47bf-947a-0526b8a5b56c service neutron] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:29:01.958562 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-c170e634-95d4-47bf-947a-0526b8a5b56c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:29:01.971871 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c170e634-95d4-47bf-947a-0526b8a5b56c service neutron] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:29:01.990128 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:29:02.027139 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1ae45e-ef22-45f0-9312-6fed0936aea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:29:02.027770 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1ae45e-ef22-45f0-9312-6fed0936aea3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:29:02.028051 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1ae45e-ef22-45f0-9312-6fed0936aea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:29:02.028351 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1ae45e-ef22-45f0-9312-6fed0936aea3 admin admin] PATH_INFO: `/v3/users/31ccfce2e43f498ab30e5d18f89e3d22` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:29:02.029529 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1ae45e-ef22-45f0-9312-6fed0936aea3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=31ccfce2e43f498ab30e5d18f89e3d22)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:29:02.031439 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1ae45e-ef22-45f0-9312-6fed0936aea3 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:29:02.198742 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-4f1ae45e-ef22-45f0-9312-6fed0936aea3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31ccfce2e43f498ab30e5d18f89e3d22'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:29:02.210868 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-4f1ae45e-ef22-45f0-9312-6fed0936aea3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31ccfce2e43f498ab30e5d18f89e3d22'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:29:02.217595 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-4f1ae45e-ef22-45f0-9312-6fed0936aea3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31ccfce2e43f498ab30e5d18f89e3d22'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:29:02.222184 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-4f1ae45e-ef22-45f0-9312-6fed0936aea3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31ccfce2e43f498ab30e5d18f89e3d22'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:29:02.222184 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-4f1ae45e-ef22-45f0-9312-6fed0936aea3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31ccfce2e43f498ab30e5d18f89e3d22'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:29:02.303878 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1461/2937] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 09:29:01 2022] DELETE /identity/v3/users/31ccfce2e43f498ab30e5d18f89e3d22 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:29:02.308430 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2519c134-020b-4966-99bf-2018a04024be None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:29:02.309269 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2519c134-020b-4966-99bf-2018a04024be None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:29:02.309494 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2519c134-020b-4966-99bf-2018a04024be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:29:02.328890 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-2519c134-020b-4966-99bf-2018a04024be None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:29:02.330167 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-2519c134-020b-4966-99bf-2018a04024be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:29:02.388703 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1477/2938] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 09:29:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:29:02.398475 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1ae45e-ef22-45f0-9312-6fed0936aea3 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:29:02.399944 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-4f1ae45e-ef22-45f0-9312-6fed0936aea3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:29:02.458045 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1ae45e-ef22-45f0-9312-6fed0936aea3 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:29:02.461548 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1ff58f-1673-41b1-a20d-29a9d20e681d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:29:02.462142 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1ff58f-1673-41b1-a20d-29a9d20e681d service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:29:02.464432 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1ff58f-1673-41b1-a20d-29a9d20e681d service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:29:02.464432 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1ff58f-1673-41b1-a20d-29a9d20e681d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:29:02.471591 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1ff58f-1673-41b1-a20d-29a9d20e681d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:29:02.473062 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1ff58f-1673-41b1-a20d-29a9d20e681d service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:29:02.482407 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1462/2939] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:29:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:29:02.888838 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2519c134-020b-4966-99bf-2018a04024be None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:29:02.890315 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-2519c134-020b-4966-99bf-2018a04024be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:29:02.902628 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2519c134-020b-4966-99bf-2018a04024be None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:29:02.909011 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6bf8ba-9af2-4852-9c19-23fcc0734ff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:29:02.909011 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6bf8ba-9af2-4852-9c19-23fcc0734ff6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:29:02.909011 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6bf8ba-9af2-4852-9c19-23fcc0734ff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:29:02.909011 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6bf8ba-9af2-4852-9c19-23fcc0734ff6 admin admin] PATH_INFO: `/v3/projects/902c69449fff4884b9ca0d955b105922` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:29:02.909981 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6bf8ba-9af2-4852-9c19-23fcc0734ff6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=902c69449fff4884b9ca0d955b105922)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:29:02.911996 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6bf8ba-9af2-4852-9c19-23fcc0734ff6 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:29:02.934737 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:29:03.012834 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-5c6bf8ba-9af2-4852-9c19-23fcc0734ff6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '902c69449fff4884b9ca0d955b105922', 'request_id': 'req-5c6bf8ba-9af2-4852-9c19-23fcc0734ff6'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:29:03.018657 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-5c6bf8ba-9af2-4852-9c19-23fcc0734ff6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '902c69449fff4884b9ca0d955b105922', 'request_id': 'req-5c6bf8ba-9af2-4852-9c19-23fcc0734ff6'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:29:03.091465 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-5c6bf8ba-9af2-4852-9c19-23fcc0734ff6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '902c69449fff4884b9ca0d955b105922', 'request_id': 'req-5c6bf8ba-9af2-4852-9c19-23fcc0734ff6'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:29:03.103249 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-5c6bf8ba-9af2-4852-9c19-23fcc0734ff6 admin admin] The token cache is being invalidate because project 902c69449fff4884b9ca0d955b105922 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 09:29:03.103775 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-5c6bf8ba-9af2-4852-9c19-23fcc0734ff6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:29:03.104892 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-5c6bf8ba-9af2-4852-9c19-23fcc0734ff6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:29:03.106418 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1478/2940] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 09:29:02 2022] DELETE /identity/v3/projects/902c69449fff4884b9ca0d955b105922 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:29:03.112068 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c5cb24-65d8-4e94-b8ff-8bbfe8b43eaf None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:29:03.112559 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c5cb24-65d8-4e94-b8ff-8bbfe8b43eaf None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:29:03.112658 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c5cb24-65d8-4e94-b8ff-8bbfe8b43eaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:29:03.131484 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-41c5cb24-65d8-4e94-b8ff-8bbfe8b43eaf None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:29:03.132683 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-41c5cb24-65d8-4e94-b8ff-8bbfe8b43eaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:29:03.172222 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:29:03.191372 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1463/2941] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 09:29:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:29:03.199135 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eb1206-6622-452b-b21f-93663237a6d1 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:29:03.199419 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eb1206-6622-452b-b21f-93663237a6d1 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:29:03.199755 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eb1206-6622-452b-b21f-93663237a6d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:29:03.219117 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-98eb1206-6622-452b-b21f-93663237a6d1 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:29:03.221316 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-98eb1206-6622-452b-b21f-93663237a6d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:29:03.229059 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1479/2942] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 09:29:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:29:03.233208 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c5cb24-65d8-4e94-b8ff-8bbfe8b43eaf None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:29:03.244551 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c5cb24-65d8-4e94-b8ff-8bbfe8b43eaf None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:29:03.251689 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de464525-99ba-4460-b805-91ec6abe9b2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:29:03.251689 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de464525-99ba-4460-b805-91ec6abe9b2d admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:29:03.251689 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de464525-99ba-4460-b805-91ec6abe9b2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:29:03.251689 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de464525-99ba-4460-b805-91ec6abe9b2d admin admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:29:03.252163 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de464525-99ba-4460-b805-91ec6abe9b2d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 09:29:03.252163 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de464525-99ba-4460-b805-91ec6abe9b2d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:29:03.253407 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de464525-99ba-4460-b805-91ec6abe9b2d admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:29:03.264805 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1464/2943] 10.222.0.57 () {58 vars in 1153 bytes} [Wed Aug 10 09:29:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 09:29:03.269208 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eb1206-6622-452b-b21f-93663237a6d1 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:29:03.278275 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eb1206-6622-452b-b21f-93663237a6d1 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:29:03.283496 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e938022-5f62-4562-bd7f-d6a1b95a6195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:29:03.284089 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e938022-5f62-4562-bd7f-d6a1b95a6195 admin admin] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:29:03.284367 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e938022-5f62-4562-bd7f-d6a1b95a6195 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:29:03.284638 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e938022-5f62-4562-bd7f-d6a1b95a6195 admin admin] PATH_INFO: `/v3/projects` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:29:03.285297 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e938022-5f62-4562-bd7f-d6a1b95a6195 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:29:03.287184 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e938022-5f62-4562-bd7f-d6a1b95a6195 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:29:03.368876 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1480/2944] 10.222.0.57 () {60 vars in 1153 bytes} [Wed Aug 10 09:29:03 2022] POST /identity/v3/projects => generated 375 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 09:29:03.373045 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de464525-99ba-4460-b805-91ec6abe9b2d admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:29:03.382898 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de464525-99ba-4460-b805-91ec6abe9b2d admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:29:03.441928 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd541f36-9862-42e7-b937-6e3d79cf6511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:29:03.442533 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd541f36-9862-42e7-b937-6e3d79cf6511 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:29:03.442813 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd541f36-9862-42e7-b937-6e3d79cf6511 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:29:03.443085 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd541f36-9862-42e7-b937-6e3d79cf6511 admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:29:03.443713 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd541f36-9862-42e7-b937-6e3d79cf6511 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:29:03.445669 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd541f36-9862-42e7-b937-6e3d79cf6511 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:29:03.577250 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1465/2945] 10.222.0.57 () {60 vars in 1144 bytes} [Wed Aug 10 09:29:03 2022] POST /identity/v3/users => generated 288 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 09:29:03.581784 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e938022-5f62-4562-bd7f-d6a1b95a6195 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:29:03.591937 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e938022-5f62-4562-bd7f-d6a1b95a6195 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:29:03.597214 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be950d70-d8eb-4d7f-9d78-c24ada2e8106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:29:03.597814 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be950d70-d8eb-4d7f-9d78-c24ada2e8106 admin admin] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:29:03.598095 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be950d70-d8eb-4d7f-9d78-c24ada2e8106 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:29:03.598366 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be950d70-d8eb-4d7f-9d78-c24ada2e8106 admin admin] PATH_INFO: `/v3/roles` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:29:03.599022 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be950d70-d8eb-4d7f-9d78-c24ada2e8106 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:29:03.600608 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be950d70-d8eb-4d7f-9d78-c24ada2e8106 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:29:03.610590 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1481/2946] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 09:29:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 09:29:03.614374 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd541f36-9862-42e7-b937-6e3d79cf6511 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:29:03.624140 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd541f36-9862-42e7-b937-6e3d79cf6511 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:29:03.628900 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238700aa-ecce-4802-9983-5754cd777009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:29:03.629511 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238700aa-ecce-4802-9983-5754cd777009 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:29:03.629795 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238700aa-ecce-4802-9983-5754cd777009 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:29:03.630065 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238700aa-ecce-4802-9983-5754cd777009 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:29:03.630697 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238700aa-ecce-4802-9983-5754cd777009 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:29:03.632300 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238700aa-ecce-4802-9983-5754cd777009 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:29:03.643075 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1466/2947] 10.222.0.57 () {58 vars in 1122 bytes} [Wed Aug 10 09:29:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 09:29:03.646961 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be950d70-d8eb-4d7f-9d78-c24ada2e8106 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:29:03.656896 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be950d70-d8eb-4d7f-9d78-c24ada2e8106 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:29:03.662196 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b762d278-f2ae-40f3-bbfd-c4dfb834f442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:29:03.662844 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b762d278-f2ae-40f3-bbfd-c4dfb834f442 admin admin] REQUEST_METHOD: `PUT` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:29:03.663151 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b762d278-f2ae-40f3-bbfd-c4dfb834f442 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:29:03.663431 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b762d278-f2ae-40f3-bbfd-c4dfb834f442 admin admin] PATH_INFO: `/v3/projects/1de216ac87914d82bd8cdf7bcc58fe10/users/a66407ade2ed49ce958e59e29e624cee/roles/44b64db484f24a46bf0b7ced46b519bf` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:29:03.678796 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b762d278-f2ae-40f3-bbfd-c4dfb834f442 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1de216ac87914d82bd8cdf7bcc58fe10, user_id=a66407ade2ed49ce958e59e29e624cee, role_id=44b64db484f24a46bf0b7ced46b519bf)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:29:03.681585 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b762d278-f2ae-40f3-bbfd-c4dfb834f442 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:29:03.749533 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1482/2948] 10.222.0.57 () {60 vars in 1483 bytes} [Wed Aug 10 09:29:03 2022] PUT /identity/v3/projects/1de216ac87914d82bd8cdf7bcc58fe10/users/a66407ade2ed49ce958e59e29e624cee/roles/44b64db484f24a46bf0b7ced46b519bf => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 09:29:03.761079 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fb151a-a6fb-4a95-af1e-bd65918499bd None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:29:03.761383 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fb151a-a6fb-4a95-af1e-bd65918499bd None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:29:03.761655 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fb151a-a6fb-4a95-af1e-bd65918499bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:29:03.789538 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-c6fb151a-a6fb-4a95-af1e-bd65918499bd None None] MFA Rules not processed for user `a66407ade2ed49ce958e59e29e624cee`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:29:03.790740 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-c6fb151a-a6fb-4a95-af1e-bd65918499bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:29:03.853318 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1467/2949] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 09:29:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:29:03.863903 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b762d278-f2ae-40f3-bbfd-c4dfb834f442 admin admin] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:29:03.865604 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-b762d278-f2ae-40f3-bbfd-c4dfb834f442 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:29:03.921357 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b762d278-f2ae-40f3-bbfd-c4dfb834f442 admin admin] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:29:03.925315 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c639e928-3925-4733-8941-1341db6d8396 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ba74e28ced4cbba857f54222dfbe41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:29:03.925916 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c639e928-3925-4733-8941-1341db6d8396 service cinder] REQUEST_METHOD: `GET` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:29:03.926197 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c639e928-3925-4733-8941-1341db6d8396 service cinder] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:29:03.926468 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c639e928-3925-4733-8941-1341db6d8396 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:29:03.935034 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c639e928-3925-4733-8941-1341db6d8396 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:29:03.936450 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c639e928-3925-4733-8941-1341db6d8396 service cinder] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:29:03.945779 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1483/2950] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:29:03 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:29:07.888664 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fb151a-a6fb-4a95-af1e-bd65918499bd None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:29:07.930740 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:29:07.951231 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fb151a-a6fb-4a95-af1e-bd65918499bd None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:29:07.956385 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eba2d6-b7e8-45c3-b02c-50d4b06120fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:29:07.957048 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eba2d6-b7e8-45c3-b02c-50d4b06120fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:29:07.957326 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eba2d6-b7e8-45c3-b02c-50d4b06120fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:29:07.957593 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eba2d6-b7e8-45c3-b02c-50d4b06120fd admin admin] PATH_INFO: `/v3/users/a66407ade2ed49ce958e59e29e624cee` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:29:07.958704 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45eba2d6-b7e8-45c3-b02c-50d4b06120fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=a66407ade2ed49ce958e59e29e624cee)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:29:07.960604 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45eba2d6-b7e8-45c3-b02c-50d4b06120fd admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:29:08.187128 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-45eba2d6-b7e8-45c3-b02c-50d4b06120fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a66407ade2ed49ce958e59e29e624cee'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:29:08.199257 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-45eba2d6-b7e8-45c3-b02c-50d4b06120fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a66407ade2ed49ce958e59e29e624cee'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:29:08.206833 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-45eba2d6-b7e8-45c3-b02c-50d4b06120fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a66407ade2ed49ce958e59e29e624cee'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:29:08.207511 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-45eba2d6-b7e8-45c3-b02c-50d4b06120fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a66407ade2ed49ce958e59e29e624cee'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:29:08.208140 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-45eba2d6-b7e8-45c3-b02c-50d4b06120fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a66407ade2ed49ce958e59e29e624cee'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:29:08.274609 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1468/2951] 10.222.0.57 () {58 vars in 1224 bytes} [Wed Aug 10 09:29:07 2022] DELETE /identity/v3/users/a66407ade2ed49ce958e59e29e624cee => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:29:08.279112 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9f564c-8148-45d8-b517-5d54a1997c23 None None] REQUEST_METHOD: `POST` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:29:08.279597 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9f564c-8148-45d8-b517-5d54a1997c23 None None] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:29:08.279692 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9f564c-8148-45d8-b517-5d54a1997c23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:29:08.298362 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.auth.core [None req-4c9f564c-8148-45d8-b517-5d54a1997c23 None None] MFA Rules not processed for user `a79bffd3ad024a2f938d1f77cfd728d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 09:29:08.300236 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-4c9f564c-8148-45d8-b517-5d54a1997c23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:29:08.341307 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:29:08.359376 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1484/2952] 10.222.0.57 () {58 vars in 958 bytes} [Wed Aug 10 09:29:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 09:29:08.369230 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eba2d6-b7e8-45c3-b02c-50d4b06120fd admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:29:08.370715 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-45eba2d6-b7e8-45c3-b02c-50d4b06120fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:29:08.429803 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eba2d6-b7e8-45c3-b02c-50d4b06120fd admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:29:08.433566 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b537241-7b57-42ec-89c4-7faf3b081a3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4863ecff21e408c9deca645b88d3727', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:29:08.434171 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b537241-7b57-42ec-89c4-7faf3b081a3e service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:29:08.434757 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b537241-7b57-42ec-89c4-7faf3b081a3e service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:29:08.435029 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b537241-7b57-42ec-89c4-7faf3b081a3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:29:08.444356 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b537241-7b57-42ec-89c4-7faf3b081a3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:29:08.445753 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b537241-7b57-42ec-89c4-7faf3b081a3e service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:29:08.455165 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1469/2953] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:29:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:29:08.859670 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9f564c-8148-45d8-b517-5d54a1997c23 None None] Authenticating user token {{(pid=80461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:29:08.861156 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.fernet_utils [None req-4c9f564c-8148-45d8-b517-5d54a1997c23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:29:08.874745 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9f564c-8148-45d8-b517-5d54a1997c23 None None] Validating token access rules against request {{(pid=80461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:29:08.879119 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24af6feb-3ad7-4768-a74a-f83637e768d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a79bffd3ad024a2f938d1f77cfd728d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe95c65d461a414fa25f77dc13eab47b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:29:08.879753 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24af6feb-3ad7-4768-a74a-f83637e768d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:29:08.880031 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24af6feb-3ad7-4768-a74a-f83637e768d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:29:08.880298 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24af6feb-3ad7-4768-a74a-f83637e768d2 admin admin] PATH_INFO: `/v3/projects/1de216ac87914d82bd8cdf7bcc58fe10` {{(pid=80461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:29:08.881432 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24af6feb-3ad7-4768-a74a-f83637e768d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1de216ac87914d82bd8cdf7bcc58fe10)` {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:29:08.883406 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24af6feb-3ad7-4768-a74a-f83637e768d2 admin admin] RBAC: Authorization granted {{(pid=80461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:29:09.020786 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-24af6feb-3ad7-4768-a74a-f83637e768d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1de216ac87914d82bd8cdf7bcc58fe10', 'request_id': 'req-24af6feb-3ad7-4768-a74a-f83637e768d2'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:29:09.026161 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-24af6feb-3ad7-4768-a74a-f83637e768d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1de216ac87914d82bd8cdf7bcc58fe10', 'request_id': 'req-24af6feb-3ad7-4768-a74a-f83637e768d2'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:29:09.115857 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-24af6feb-3ad7-4768-a74a-f83637e768d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1de216ac87914d82bd8cdf7bcc58fe10', 'request_id': 'req-24af6feb-3ad7-4768-a74a-f83637e768d2'} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:29:09.128594 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-24af6feb-3ad7-4768-a74a-f83637e768d2 admin admin] The token cache is being invalidate because project 1de216ac87914d82bd8cdf7bcc58fe10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 09:29:09.128880 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-24af6feb-3ad7-4768-a74a-f83637e768d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:29:09.129553 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG keystone.notifications [None req-24af6feb-3ad7-4768-a74a-f83637e768d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 09:29:09.131059 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: [pid: 80461|app: 0|req: 1485/2954] 10.222.0.57 () {58 vars in 1233 bytes} [Wed Aug 10 09:29:08 2022] DELETE /identity/v3/projects/1de216ac87914d82bd8cdf7bcc58fe10 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 09:29:18.450910 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=61,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:29:19.124904 dv-cinder-780759-5-iscsi devstack@keystone.service[80461]: DEBUG dbcounter [-] [80461] Writing DB stats keystone:SELECT=33,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 09:31:13.314686 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b537241-7b57-42ec-89c4-7faf3b081a3e service neutron] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 09:31:13.316024 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-6b537241-7b57-42ec-89c4-7faf3b081a3e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:31:13.426221 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b537241-7b57-42ec-89c4-7faf3b081a3e service neutron] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 09:31:13.484521 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b94b28d-4f57-4c05-bfb4-a94b0c03cf66 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2912307c56594125b157f1cfdf7c96a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49e381419e940278bcb6111899589b5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 09:31:13.485145 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b94b28d-4f57-4c05-bfb4-a94b0c03cf66 service placement] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 09:31:13.485427 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b94b28d-4f57-4c05-bfb4-a94b0c03cf66 service placement] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 09:31:13.485695 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b94b28d-4f57-4c05-bfb4-a94b0c03cf66 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 09:31:13.487402 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-6b94b28d-4f57-4c05-bfb4-a94b0c03cf66 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 09:31:13.500333 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b94b28d-4f57-4c05-bfb4-a94b0c03cf66 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 09:31:13.501986 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b94b28d-4f57-4c05-bfb4-a94b0c03cf66 service placement] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 09:31:13.512627 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1470/2955] 10.222.0.57 () {60 vars in 1365 bytes} [Wed Aug 10 09:31:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 09:31:23.507667 dv-cinder-780759-5-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=75 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}